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Preface 

Nowadays, production systems have become more complex since they require adapting 
to the market challenges, to introduce more flexible mechanisms and control strategies in 
their structure to ensure the efficient use of system resources and to allow system layout 
reconfiguration. 

The book presents an excellent overview of the recent developments in the different 
areas of Robotics, Automation and Control. Through its 24 chapters, this book presents 
topics related to control and robot design; it also introduces new mathematical tools and 
techniques devoted to improve the system modeling and control. An important point is the 
use of rational agents and heuristic techniques to cope with the computational complexity 
required for controlling complex systems. Through this book, we also find navigation and 
vision algorithms, automatic handwritten comprehension and speech recognition systems 
that will be included in the next generation of productive systems developed by man. 

We would like to thank all the authors for their excellent contributions in the different 
areas of the control, robotics and automation. It is their knowledge and enthusiastic 
collaboration that lead to the creation of this book, which we are sure that will be very 
valuable to the readers. 

October 2008 

Editors 

Jesus Aramburo 
Antonio Ramirez Trevino 
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1. Introduction 

In traditional Computer Numerical Control (CNC) systems, machining parameters are 
usually selected prior to machining according to handbooks or user's experience. These 
practices tend to select conservative parameters in order to avoid machining failure and 
assure product quality specifications. Less conservative practices try to find optimal 
machining parameters off-line to increase process productivity after conducting 
experimentation (Chien & Chou, 2001). However, variations during the machining process 
due to tool wear, temperature changes, vibrations and other disturbances make inefficient 
any off-line optimization methodology, especially in high quality machining operations 
where product quality specifications are very restrictive. Therefore, to assure the quality of 
machining products, reduce costs and increase machining efficiency, cutting parameters 
must be optimised in real-time according to the actual state of the process. This optimization 
process in real-time is conducted through an adaptive control of the machining process. 
The adaptive control applied in machining systems is classified as (Liang et al., 2004; Ulsoy 
& Koren, 1989): Adaptive Control with Constraints (ACC), Geometric Adaptive Control 
(GAC), and Adaptive Control with Optimization (A CO). In the ACC systems, process 
parameters are manipulated in real time to maintain a specific process variable, such as 
force or power, at a constraint value. Typically, ACC systems are utilized in roughing 
operations where material removal rate is maximized by maintaining the cutting forces at 
the highest possible cutting force such that the tool is not in danger of breaking (Zuperl et 
al., 2005). In the GAC systems, the economic process optimization problem is dominated by 
the need to maintain product quality such as dimensional accuracy and/ or surface finish 
(Coker & Shin, 1996). GAC systems are typically used in finishing operations with the 
objective of maintaining a specific part quality despite structural deflections and tool wear. 
Sensor feedback is often employed to measure surface roughness and dimensional quality 
between parts and adjustments, so tool offsets and feed overrides can be adjusted for the 
next part. In the ACO systems, machine settings are selected to optimize a performance 
index such as production time, unit cost, etc. Traditionally, ACO systems have dealt with 
adjusting cutting parameters (feed-rate, spindle speed and depth of cut) to maximise 
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material removal rate subject to constraints such as surface roughness, power consumption, 
cutting forces, etc (Venu Gopal & Venkateswara Rao, 2003). Other ACO systems optimise a 
multi-objective function which are more practical in industrial applications (Zuperl & Cus, 
2005). For example, it is quite often to search the optimal cutting parameters to minimize the 
cost of the operation, maximize the production rate and maximize the part quality. ACO 
systems are basically composed of several units which integrate the machine-tool system 
and the equipment required for acquiring real-time process measurements and adjusting the 
cutting parameters. Fig. (1) shows a simplified scheme of a basic ACO system presented in 
(Koren, 1983). Basically, the ACO system requires a sensor system which provides real-time 
data for tool wear diagnosis and part quality prediction. The real-time data are used by 
process models previously obtained from experimental data. Tool wear and part quality 
models are used in the multi-objective function together with cutting parameters. An 
optimizer unit is then applied for searching optimal cutting parameters, and the selected 
parameters are sent to the CNC system. 
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Fig. 1. Adaptive Control Optimization (ACO) scheme adapted from (Koren, 1983). 

Interesting works related to ACO systems can be found in (Liu & Wang, 1999; Liu et al., 
1999; Chiang et al., 1995). Liu (Liu & Wang, 1999) proposed an adaptive control system 
based on two neural network models, a Back-Propagation Neural Network (BP NN) and an 
Augmented Lagrange Multiplier Neural Network (ALM NN). The BP NN was used for 
modeling the state of the milling system, using as a single input the feed parameter and 
sensing the cutting forces on-line. The ALM NN was used for maximising the material 
removal rate which it was carried out adjusting the feed rate. Chiang (Chiang et al., 1995) 
presented a similar work for end-milling operations, but surface roughness was also 
considered as constraint. Both research works were based on theoretical formulas for 
training the neural networks, and both applied an ALM NN for optimization, which it is 
claimed to be an approach that can greatly reduce processing time in comparison to 
conventional optimal algorithms and make real-time control possible. Liu (Liu et al., 1999) 
also extended his previous work with a new optimization procedure based on a Genetic 
Algorithm (GA). 
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In spite of the potential application of ACO systems, their use in industry is limited due to 
the non-existence of reliable on-line monitoring systems for tool wear diagnosis and quality 
prediction (Azouzi & Guillot, 1997; Liang et al., 2004). Therefore, the optimal selection of 
cutting parameters is usually done off-line for the cutting-tool life-cycle (Ghani et al., 2004; 
Chien & Chou, 2001). The off-line parameters optimization is usually carried out through 
short cutting experiments which are later used to obtain an empirical model which could be 
optimized subjected to some constraints. Ghani (Ghani et al., 2004) optimized cutting 
parameters using a Taguchi's Design of Experiments in end milling operations. With a 
minimum number of trials compared with other approaches such as a full factorial design, 
the methodology presented reveals the most significant factors and interactions during 
cutting process which leads to choose optimal conditions. A similar methodology is 
described in (Zhang et al., 2007). However, both methodologies do not permit to evaluate 
quadratic or non-linear relations between factors, and the analysis is restricted to the levels 
analysed in each factor. A more generic approach although more costly in experiments is 
based on Response Surface Model (RSM) and Response Surface Model Optimization 
(RSMO). Suresh (Suresh et al., 2002) used RSM for modeling the surface roughness as a first 
and second-order mathematical model and the surface roughness optimization was carried 
out through GA. Cus (Cus & Balic, 2003) also applied GA for optimising a multi-objective 
function based on minimum time necessary for manufacturing, minimum unit cost and 
minimum surface roughness. All the process models applied in his research were empirical 
formulas from machining handbooks which were fitted through regressions. More complex 
models have also been applied for surface roughness and tool wear modeling to optimise 
off-line cutting parameters. Zuperl (Zuperl & Cus, 2003) also applied and compared feed- 
forward and radial basis neural networks for learning a multi-objective function similar to 
the one presented in (Cus & Balic, 2003). Choosing the radial basis networks due to their fast 
learning ability and reliability, he applied a large-scale optimization algorithm to obtain the 
optimal cutting parameters. Chien (Chien & Chou, 2001) applied neural networks for 
modeling surface roughness, cutting forces and cutting-tool life and applied a GA to find 
optimum cutting conditions for maximising the material removal rate under the constraints 
of the expected surface roughness and tool life. 

These previous works are off-line optimization methodologies which can be efficient 
enough if tool wear effects have a minimal impact to surface roughness and/ or a high 
surface roughness quality is not required. Otherwise, an on-line optimization methodology 
should be applied since optimal cutting conditions may vary during the cutting-tool life- 
cycle due to tool wear effects on surface roughness. In this chapter, an ACO system is 
presented for optimising a multi-objective function based on material removal rate, quality 
loss function related to surface roughness, and cutting-tool life subjected to surface 
roughness specifications constraint. The proposed system adjusts the cutting parameters 
during the cutting-tool life-cycle in order to maximise in real-time the multi-objective 
function. The core of the system is composed of three process models: a cutting-tool wear 
model for diagnosing the state of the cutting tool, a surface roughness deviation model for 
predicting the quality loss function and a cutting-tool life model. All models are developed 
using artificial neural networks to model the non-linear relationships in machining 
processes. Since the process models are black-box models, optimal cutting parameters are 
obtained applying genetic algorithms and mesh adaptive direct search algorithms. The 
proposed system is compared with 2 traditional methods for off-line cutting parameters 
selection: (1) selection based on suggested cutting parameters from handbooks, and (2) 
selection based on RSMO. 
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2. Experimental system 

2.1 Machining process description 

Machining hardened steels (hardness from 30 to 62 HRC) for moulds and dies with surface 
roughness specifications less than 0.3 microns are commonly applied in industry, and 
require costly and time-consuming traditional operations such as electro-discharge 
machining or grinding. Recently, some research studies have reported the use of high 
performance machining operations for these applications with important benefits as 
reducing lead times and costs (Siller et al., 2008). However, tool wear process impacts 
directly to surface roughness so optimal cutting parameters are difficult to obtain since they 
vary according to cutting-tool state. Therefore, although high performance machining can 
technically substitute grinding or electro-discharge machining, additional efforts should be 
conducted in order to tune cutting parameters for an optimal machining. For these 
applications, A CO techniques can improve the process significantly with respect to other 
non-adaptive optimization techniques. 

The machining process studied in this paper is presented in Fig. 2, and it consists of a face- 
milling operation on workpieces of hardened AISI D3 steel (60 HRc) with dimensions 
250x250 mm. The experiments were conducted on a CNC machining center suited for 
mould and die manufacturing, and the cutting tool used was a face milling tool with Cubic 
Boron Nitride (CBN) inserts. In order to generate a good surface finish and avoid run-out 
problems, a single insert was mounted on a tool body with an effective diameter of 6.35 mm. 



,ll 


1 t li^n 





Fig. 2. Machining process analysed 



2.2 Monitoring system description 

A monitoring system to estimate on-line tool wear and surface roughness is required to 
select the optimal cutting parameters according to the actual state of the machining process. 
In this chapter, the monitoring system implemented is a multi-component sensor system 
composed of a piezoelectric dynamometer, accelerometers and signal conditioners (Fig. 3). 
Two acquisition boards were used for data acquisition. The first board, an Iotech DaqBook 
112, was used for acquiring cutting forces from the dynamometer and it was configured for 
a sample frequency of 3 kHz. A second board, an Iotech DaqBoard 3000, was used for 
vibration signal acquisition from accelerometers and it was configured for a sample 
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frequency of 100 kHz. Cutting forces were amplified and filtered by a Kistler 5405 amplifier 
configured with a low-pass filter of 300 Hz cut-off frequency. Vibration signals were 
amplified by a PCB 482A22 amplifier. Root-mean-square of forces and vibrations were 
calculated for each cutting pass at the cutting-location x = 175 mm for a 2 seconds data 
acquisition. Surface roughness (Ra) was measured by a Mitutoyo Surftest 301 profilometer 
at the cutting-tool locations x = 40 mm, x = 110 mm, x = 175 mm every cutting pass 
(sampling length X = c/1 = 0.8 mm and number of spans n = 5). Cutting tool wear (Vb) was 
measured by a stereo-microscope Nikon MZ12 after each face-milling pass every 250 mm 
length of cut. Fig. 4 describes the machining process with the Ra and Vb sampling 
procedure. 




Fig. 3. Multi-component sensor system 




Vb Measurement 
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Fig. 4. Machining process and surface roughness and tool- wear sampling. 



3. Design of experiments 

In order to compare cutting parameters optimization by RSMO and AI approaches, it is 
necessary to carry out a Design of Experiments (DoE) to be useful for both. RSMO requires 
classical designs of experiments such as Box- Wilson Central Composites Designs (CCD) or 
Box-Behnken designs (Nist, 2006), in case that it is only considered linear and quadratic 
effects. On the other hand, AI approaches require enough data for training and testing, 
varying the factors in all its domain, but it does not require any specific DoE design. 
The factors considered in the experimentation were the feed per tooth (f z ) and the cutting 
speed (V c ). The radial depth of cut (a e ) was considered constant, with a value of 31.25 mm to 
maximize the material removal rate. The axial depth of cut (a p ) was defined as a constant 
(0.4 mm) since the machining operation studied was a finishing operation. The minimal 
experimentation required to apply RSMO with two factors is a face centered CCD with one 
center point which is equivalent to a 2 3 full factorial design. For each experiment, the face- 
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milling operation was carried out until the cutting tool edge was worn (Vb higher than 0.3 
mm, usual value for finishing operations (ISO 8688-1, 1989)) or the surface roughness was 
outside specifications. Fig 5 shows the cutting conditions analysed and the order of the 
cutting experiments. 
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Fig. 5. Design of Experiments and run order. Face Centered Central Composite Design. 
4. Definition of the optimization problem 

The machining economics problem consists in determining the optimal cutting parameters 
in order to maximize/ minimize an objective function. Typical objective functions to 
optimize cutting parameters are " minimize unit production cost", " maximize production 
rate", "maximize profit rate", etc. On the other hand, several cutting constraints have to be 
considered in machining economics, such as tool-life constraint, cutting force constraint, 
power, stable cutting region constraint, chip-tool interface temperature constraint and 
surface finish constraint (Cus & Balic, 2003). 

4.1 Objective functions 

Typically, three objective functions are considered in a cutting parameters optimization 
problem: (1) Material Removal Rate (MRR), (2) surface roughness and (3) cutting-tool life. 
MRR is a measurement of productivity, and it can be expressed by analytical derivation as 
the product of the width of cut (w), the feed velocity of the milling cutter (F) and the depth 
of cut (a p ) (Eq. (1)). Surface roughness is the most important criterion for the assessment of 
the surface quality, and it is usually calculated empirically through experiments. Some 
research works directly use the empirical relationship presented in Eq. (2), where Vc and f 
are the cutting speed and feed rate respectively and k, xi, X2, X3 are empirical coefficients. 
Cutting-tool life is the other important criterion for cutting parameters selection, since 
several costs such as cutting-tool replacement cost and cutting-tool cost are directly related 
with tool life. The relation between the tool life and the parameters is usually expressed by 
the well-known Taylor's formula presented in Eq. (3), where Kt, a\, 0:2, 0:3 are empirical 
coefficients. 



MRR = wa p F 



Ra = k V c Xl f x * a/s 



T = 



Vc* 1 f* 2 a p K * 



(1) 
(2) 

(3) 



Adaptive Control Optimization of Cutting Parameters for High Quality Machining Operations 

based on Neural Networks and Search Algorithms 7 

However, for high quality machining operations using CBN cutting tools, both traditional 
surface roughness and tool life equations may not provide a good estimation. Machining a 
very low feed speeds produce that additional mechanisms influence the surface roughness 
generation such as vibrations, engagement of the cutting tool, built up edge, etc. (Siller et al., 
2008). On the other hand, CBN tools have a different wear process than traditional cutting- 
tools such as high speed steels, so Taylor's formula may not be directly applied (Trent & 
Wright, 2000). For both reasons, other empirical models based on experimental data must be 
applied instead of Eqs. (2,3). 

For the case study presented in this chapter which is a high quality face milling operation 
based on CBN tools, two alternative objective functions were applied. Instead of Ra model, 
it is applied the quality loss function described by Eq. (4). Considering a desired Ra value, 
the quality loss function is usually applied to estimate the cost of manufacturing with a 
quality variation. The loss function is defined as: 

V 2 

W = ^rework ^ ( 4 ) 

where A = Ra max - Ra ta rget with Ra max the maximum Ra defined by specifications and Ra ta rget 
the Ra desired; V 2 is the mean squared deviation as V 2 = ((Ra targ et - yi) 2 + . . . + (Ra targ et - 
y n ) 2 )/n , with n the number of samples; and A reW ork is the part cost if the part is outside 
specifications. On the other hand, instead of the traditional Taylor's formula, it is applied an 
empirical model learnt from the experimentation which is defined by the Eq. (5), where f is 
the function learnt. 

T = f(V € ,f,a p ) (5) 



4.2 Multi-objective function 

The optimization problem for the case study is defined as the optimization of a multi- 
objective function which is composed of the objective functions defined by Eqs (1,4,5). Since 
these objective functions are conflicting and incomparable, the multi-objective function is 
defined using the desirability function approach. This function is based on the idea that the 
optimal performance of a process that has multiple performance characteristics is reached 
when the process operates under the most desirable performance values (Nist, 2006). For 
each objective function Yi(x), a desirability function di(Yi) assigns numbers between and 1 
to the possible values of Yi, with di(Yi) = representing a completely undesirable value of Yi 
and di(Yi) = 1 representing a completely desirable or ideal objective value. Depending on 
whether a particular objective function Yi is to be maximized or minimized, different 
desirability functions di(Yi) can be used. A useful class of desirability functions was 
proposed by (Derringer & Suich, 1980). Let Li and Ui be the lower and upper values of the 
objective function respectively, with Li < Ui, and let Ti be the desired value for the objective 
function. Then, if an objective function Yi(x) is to be maximized, the individual desirability 
function is defined as 



d,«) = 



IfYi(x)<Li 



(tFIt) Vh*YMST t \ (6) 

1 IfY l (x)>T i 
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with the exponent w is a weighting factor which determines how important it is to hit the 
target value. For w = 1, the desirability function increases linearly towards Ti; for w < 1, the 
function is convex and there is less emphasis on the target; and for w > 1, the function is 
concave and there is more emphasis on the target. If one wants to minimize an objective 
function instead, the individual desirability function is defined as 



1 // 7,0X7; 

(Yi ~ U{ 



IfY i (x)>U i 



(7) 







Fig. (6) shows the individual desirability functions according to different w values. The 
individual desirability functions are combined to define the multi-objective function, called 
the overall desirability of the multi-objective function. This measure of composite 
desirability is the weighted geometric mean of the individual desirability for the objective 
functions. The optimal solution (optimal operating conditions) can then be determined by 
maximizing the composite desirability. The individual desirability is weighted by 
importance factors Ii. Therefore, the multi-objective function or the overall desirability 
function to optimize is defined as: 



(8) 



D = (fkOQPdtQtf* ...d k (Y k y k )(n + i2 + - + ,k) 



with k denoting the number of objective functions and Ii is the importance for the objective 
function I, where i = 1,2,. . .,k. 
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Fig. 6. Desirability functions according to the type of objective function 



4.3 Constraints 

Due to the limitations on the cutting process, manufacturers limit the range of the cutting 
parameters to avoid premature cutting-tool failures. Therefore, selected cutting parameters 
according to manufacturer specifications are constrained to: 



Vmin — V c ^ Vmax 
Imin — Iz — Imax 



(9) 
(10) 
(11) 
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Surface roughness specification is also considered a constraint that can be expressed as 

R a < R sp ec ■* V 2 < (Ratarget - Ra S pec) 2 (12) 

In addition, cutting power and force limitations are usual constraints, but they are 
commonly applied only for roughing operations. 

4.4 Summary of optimization problem and numerical coefficients 

The weights and the individual desirability coefficients for each objective function were 
chosen according to each objective function in the machining process. First the weights were 
defined considering how the objective function increases/ decreases as the ideal value is not 
matched. Secondly, a comparison among individual desirability coefficients was done to 
define how much more important is each objective function than the other one. For the case 
study presented, the objective functions were considered linear (w=l) and the coefficient of 
importance were chosen to prevail productivity and surface roughness quality than cutting- 
tool cost and cutting-tool cost replacements. Therefore, importance factors Ii and h which 
are related to material removal rate and surface quality loss function were chosen as 1, 
whereas importance factor I3 which is related to cutting-tool life was chosen as 0.5. 
Considering the maximum and minimum values of each objective function obtained 
analytically, the desirability functions were defined as follows. 
MRR desirability function 

MRR - 398 , x 

MRRtarget = 2387 mm 3 /min. MRR m inimum = 398 mm 3 /min. Importance factor Ii=l. 
Desirability function of Ra deviation objective function 

V 2 - 012 
d 2 (W) = d 2 (V 2 ) = ft/wwv , ' n (14) 

2 2 0.0001-0.012 

V 2 tar g et = 0.0001 |um 2 . V 2 m aximum = 0.012 |um 2 . Importance factor 12=1. Note that the desirability 
function of quality loss W for surface roughness can be defined by the surface roughness 
deviation V 2 since Eq. (4) relates W with V 2 by a constant coefficient of A reW ork/ A 2 . 
Cutting-tool life desirability function 

T — 7 43 

d 3 (T)=- -^- (15) 

3V } 46.7-7.43 V ' 

Ttarget = 46.7 min. T m i n i mum = 7.43 min. Importance factor 13=0.5. 

The multi-objective function or the overall desirability function to be optimized is: 



constrained to: 



D = (d 1 (Mi?7?) 1 d 2 (K 2 ) 1 d 3 (r)°- 5 )(i+i+o.5) (16) 

100 m/min < V c < 200 m/min (17) 

0.04 mm/ rev < f z < 0.12 mm/ rev (18) 
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a p = 0.4 mm 



V2 < (0.1 - 0.2)2 ^ m 2 



(19) 
(20) 



5. Parameter optimization based on handbooks 

5.1 Description 

Cutting tool parameters are traditionally chosen according to handbooks and cutting-tool 
data catalogs. For a given cutting-tool and workpiece material, a range of possible cutting- 
parameters are provided. The machinist chooses the parameters within the ranges using 
some well-known practices in shop-floor. Some of these practices are: 

Higher cutting speeds increase surface roughness quality but decrease cutting tool life. 

Higher cutting speeds decrease cutting tool life. 

Higher feed rates increase productivity as material removal rate is increased. 

Higher feed rates decrease surface roughness quality. 

Higher feed rates decrease cutting-tool life. 

Higher axial depth of cut increases productivity. 

Higher axial depth of cut decreases cutting-tool life. 

Very low axial depth of cut burns the workpiece surface and generates a low surface 

roughness quality and decreases cutting-tool life. 
According to the final goal of the machining process, the machinist selects the best cutting- 
tool parameters combination. For example, if the only important constraint is a high cutting 
tool life, the machinist will select a low cutting speed, low feed rate and low-medium axial 
depth. Fig. 7a describes the typical optimization process based on handbooks. 



Product Specifications 




Rules 



Higher Vc Then I ewer Ra 
Higher fz Then higher Ra 
_| Higher fz Then higher MRR 
Higher Vc Then lower Tool life 



Machining 

Handbooks and 

Catalogs 



Optimal Parameters 



Experimentation 




p E Ra and Vb measurements 

(Face Centered Central X 

Composite Design) 



Response Surface Models 




Product Specifications 



Multi -objective Optimization 
Optimal Parameters 



(a) OPTIMISATION BASED ON HANDBOOKS 



(b) OPTIMISATION BASED ON RSMO 



Fig. 7. Off-line cutting parameters optimization. 



5.2 Optimization results 

As it was explained above (section 4.4), the MRR is the most important objective function 
together with product quality whereas tool life is less important (Ii=l; 12=1; 13=0.5). A high 
feeding value increases MRR but decreases surface roughness, so it seems reasonably to fix 
the feeding value to an intermediate value. On the other hand, as V c increases, both MRR 
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and part quality increase, but cutting tool life decreases considerably. As both MRR and part 
quality are much important than cutting tool life, it seems reasonably to fix V c at its 
maximum value. Therefore, the optimal cutting parameters based on catalogs and 
handbooks can be defined as V c =200 m/min and fz=0.08 mm. Experimentation was 
conducted in order to check the overall desirability function at these cutting conditions. The 
experimental results showed a cutting tool life T=10.8 min, an average surface roughness 
deviation of V 2 =1.6 -lO 3 |um 2 , and a MRR=1592 mm 3 / min. The overall desirability was 0.472, 
and the evolution of the overall desirability function due to surface roughness variability 
along cutting tool life-cycle is shown in Fig (8). 




Fig. 8. Overall desirability function along cutting tool life-cycle. Parameter optimization 
based on handbooks. Vc=200m/min; fz=0.08mm; ap=0.4mm 

6. Parameter optimization based on RSMO 

6.1 Description 

A less conservative method to optimize cutting parameters could be carried out through the 
Response Surface Model Optimisation methodology (RSMO). RSMO is a collection of 
mathematical and statistical techniques that are useful for the modeling and analysis of 
problems in which a response of interest is influenced by several variables and the objective 
is to optimize this response (Montgomery, 2001). In most RSM problems, the form of the 
relationship between the variable to be optimized (the response) and the independent 
variables is unknown. Thus, the first step in RSM is to find a suitable approximation for the 
true functional relationship between the response and the set of independent variables. First 
and second-order models are commonly applied to approximate the function through least 
squares. Considering a response surface y defined by y=f(xi,X2...Xk) where xi, X2, ..., Xk are 
independent variables, the optimal value of the response surface will be the set of xi, X2, ..., 
Xk for which the partial derivatives — = — = — = 0. This point is called the stationary 

x dx x dx 2 dx k x J 

point and it could represent a point of maximum response, a point of minimum response, or 
a saddle point. A general mathematical solution for the location of the stationary point can 
be obtained as follows. Given a second-order model in matrix notation as: 



y = p +x'b + x f Bx 



(21) 



where 



x — 



Xi 


X — 


h 


UfeJ 




[Pk\ 



and B = 



Pit 2 



sym 



0ik 

2 

fak 
2 

Pkk 



(22) 



12 Advances in Robotics, Automation and Control 

That is, b is a (k x 1) vector of the first-order regression coefficients and B is a (k x k) 
symmetric matrix whose main diagonal elements are the pure quadratic coefficients 

( p it ) and whose off -diagonal elements are one-half the mixed quadratic coefficients ( /?. . , i ¥* j). 
The derivative of y with respect to the elements of the vector x equated to is 

|2 = b + 2Bx = (23) 

The stationary point is the solution to Eq. (23), or 

x s = -\B^b (24) 

To define whether the stationary point is a point of maximum or minimum response or 
saddle point, it is usually examined a contour plot of the fitted model. Refer to 
(Montgomery, 2001) for RSMO concepts. 

In this section the RSMO methodology is applied to the high quality machining operation 
studied. The experimentation conducted through the DoE is used to obtained first and 
second order process models. The multi-objective function which is based on these models 
is optimized and the optimal cutting parameters are defined. Fig. 7b describes the 
optimization process based on RSMO in the machining process studied. 

6.2 Machining process models based on first and second-order functions 

RSM technique lets model the responses of interest W, T, and MRR as a first and second- 
order functions. After conducting the DoE shown in Fig. (5), a RSM was fitted for each 
response. The first and second order functions with its coefficient of determination were: 
Ra deviation (V 2 ) response: 

V 2 = 0.0629 - 5 • W-% - 0.507/ z + 11.9 ■ lQ-% 2 + 1.49// + 1.6 • 10" 3 F C f z 

R 2 ad j =891% (25) 



T response: 

= 11/. 3 — U.36LV C — LZ>/ &.d] z T DD^D.DJ^ 

R 2 adj=80.4% 
MRR response 



T = 117.3 - 0321V C - 1372.3/ z + 5333.3// + 2.037 V c f z 



MRR = 99.47 V c f z (27) 

R 2 a dj=100%. Note that this is the exact analytical equation for MRR. 
Fig. 9 shows the response surface models for each response analysed. 

6.3 Optimization results 

The optimum cutting parameters were obtained maximizing the overall desirability 
function, which depends on the previous first and second order models and the individual 
desirability and weights coefficients defined in section 4.4. The result of the optimization 
procedure showed the optimal cutting parameters V=165m/min and fz=0.12mm, with an 
overall desirability value of 0.61. To validate the RSMO result, experimentation was 
conducted with the expected optimal parameters. The experimental results showed a 
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cutting tool life T=13.7 min, a surface roughness deviation function of Y 2 =5.5 10 3 |im 2 , and a 
MRR=1967 mm 3 / min. The overall desirability of the experiment was 0.495, and the 
evolution of the overall desirability function due to surface roughness variability along 
cutting tool life-cycle is shown in Fig (10). Note that the expected overall desirability was 
0.610 whereas the real value was 0.495. This error is due to the inaccuracy of the process 
models developed through this methodology. 




Fig. 9. Response Surface Models for deviation of Ra, T and MRR. Units: Vc (m/min); fz 
(mm); Ra deviation (\im 2 ); T (min); MRR (mm 3 / min). 




123456789 10 1112 13 1415 16 17 18 19 20 2122 23 24 25 
Cutting pass 



Fig. 10. Overall desirability function along cutting tool life-cycle. Parameter optimization 
based on RMSO. Vc=164m/min; fz=0.12mm; ap=0.4mm. 



7. Parameter optimization based on Al 

7.1 Description 

Due to non-linearity, first and second order models from response surface methodology 
cannot be enough accurate to model tool-wear and surface roughness in machining 
processes. To overcome this limitation, AI techniques such as Artificial Neural Networks or 
Fuzzy Logic can be applied in order to deal with non-linearity. However, these AI models 
are called black-box models, and they cannot be optimized with conventional optimization 
methods. Due to this limitation, a cutting parameter optimization methodology based on AI 
models requires an advanced search methods for global optimization, such as Genetic 
Algorithms (GA) and Mesh Adaptive Direct Search (MADS) algorithms. 
GAs are search algorithms based on the mechanics of natural selection and natural genetics, 
invented by (Holland, 1975), which can find the global optimal solution in complex 
multidimensional search spaces. A population of strings, representing solutions to a 
specified problem, is maintained by the GA. The GA then iteratively creates new 
populations from the old by ranking the strings and interbreeding the fittest to create new 
strings. So in each generation, the GA creates a set of strings from the previous ones, 
occasionally adding random new data to keep the population from stagnating. The end 
result is a search strategy that is tailored for vast, complex, multimodal search spaces. GAs 



14 



Advances in Robotics, Automation and Control 



are a form of randomized search, in the way in which strings are chosen and combined is a 
stochastic process. This is a radically different approach to the problem solving methods 
used by more traditional algorithms, which tend to be more deterministic in nature, such as 
the gradient methods used to find minima in graph theory. However, although GA is an 
effective optimization algorithm, it usually takes a long time to find an optimal solution due 
to its slow convergence speed (Cus & Balic, 2003). 

On the other hand, MADS algorithms are iterative search algorithms where the optimization 
is conducted through an adaptive mesh of points where the objective function is evaluated. 
At the first iteration, the mesh is built according to an initial point of the objective function. 
The algorithm computes the objective function at the mesh points until it finds one whose 
value is smaller than the objective function evaluated on the initial point. If a mesh point has 
a smaller value, the algorithm sets the next point in the sequence equal to this one and 
multiplies the current mesh size by a mesh expansion factor. The mesh is then expanded 
and the algorithm conducts a new iteration. In case none of the mesh points has a smaller 
objective function value than the value at the best current solution, the algorithm does not 
change the current point at the next iteration and the mesh size is contracted by a mesh 
contraction factor. After the re-size, a new iteration is conducted. The iterations are 
conducted until a stop condition is reached, typically when the mesh size reaches a 
minimum value. Refer to (Audet & Dennis, 2004) for concepts related to MADS. 
In this section it is proposed an optimization methodology based on Artificial Neural 
Networks (ANN) models for modeling the machining process and GA-MADS algorithms to 
optimize the multi-objective function defined by Eq. (16) and the ANN models. The 
combination of GA and MADS algorithms lets reduce the computing time required for the 
optimization. Basically, GA is firstly applied in order to find the region where the multi- 
objective function is minimum. Then, the GA algorithm is interrupted and the MADS 
algorithm refines the search using the GA solution as the initial point of the mesh. The 
optimal cutting parameters are calculated when the MADS algorithm reaches the minimum 
mesh size. Fig. 11 describes the procedure of the optimization methodology proposed, based 
on process models using AI techniques and an initial Design of Experiments, and a GA- 
MADS optimization. Table (1,2) defines the main characteristics of the ANN models applied 
and the GA-MADS algorithms. 




OPTIMISATION BASED ON AI PROCESS MODELS AND GA-MADS OPTIMISATION 



Fig. 11. On-line cutting parameters optimization based on AI 
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7.2 Machining process models based on Al 

After conducting the DoE, an experimental database composed of 188 samples of 9 different 
cutting conditions was generated. Each sample was defined by: surface roughness deviation 
(V 2 ), cutting tool wear state (% of use), root-mean-square of cutting forces in X, Y direction 
(RMS Fx, RMS Fy), root-mean-square of cutting forces in XY plane (RMS Fxy), root-mean- 
square of vibrations in X direction (RMS Ax), root-mean-square of vibrations in Y direction 
(RMS Ay), cutting time (Tc), cutting speed (Vc) and feed per tooth (fz). The experimental 
database was used to learn three ANN process models: (1) Ra deviation model, (2) Cutting- 
tool life model, (3) Cutting-tool wear state model. Before training the ANN, a statistical 
study was conducted for each model in order to discard those inputs variables that were not 
significant. The final inputs variables applied for each model and the main characteristics of 
the ANN models are shown in Table 1. Note that vibrations signals were discarded for all 
models since they were close related to the cutting-tool position, and they cannot provide 
any objective information about the state of the process. Fig 12 shows the response of each 
ANN model according to V c and f z values. 
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Table 1. Characteristics of ANN models 




Fig. 12. ANN models of : (1) surface roughness deviation -V 2 - for a cutting tool wear of 50%; 
(2) cutting tool life -T-; (3) tool wear state for a RMS Fxy of 60N. 
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7.2 Optimization results 

The optimum cutting parameters were obtained maximizing the overall desirability 
function every cutting pass of 250 mm length of cut. During the cutting pass, the cutting 
forces were acquired and the cutting-tool wear state was predicted by the cutting-tool wear 
state model. With the cutting-tool state prediction, the GA-MADS optimization procedure 
was conducted to maximize the overall desirability function, which is based on the MRR 
model, and the ANN models of Ra deviation and cutting-tool life. The optimal solution was 
obtained after an average processing time of 6 seconds, and the new cutting parameters 
were sent to the CNC controller for the next cutting pass. The process was repeated until the 
cutting tool was worn or the surface roughness was outside specifications. The result of the 
optimization procedure showed that the optimal cutting parameters vary considerably, 
from 200m/ min to 130m/ min and from 0.12mm to 0.07mm for cutting speed and feed rate 
respectively, with an expected overall desirability value of 0.545. The real overall desirability 
value after measure surface roughness deviation and tool-life was 0.520. The evolution of 
the overall desirability function and the variation of the optimal cutting conditions along the 
cutting tool life-cycle are shown in Fig (13). 
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Table 2. Characteristics of search algorithms 



8. Results and discussion 

The results reported in the optimization methodology based on handbooks and catalogs 
show the most conservative cutting parameters as it was expected, where the overall 
desirability function has a value of 0.472. A simple RSMO through a 9 experimental runs 
provide enough information to improve cutting parameter selection and increase the overall 
desirability function to 0.495 which is an improvement of 5% in the desirability. The surface 
response models show that high cutting speeds increase surface roughness variability which 
were not taken into account by the machinist in the first optimization methodology. 
Although RSMO increases the overall desirability, the RSMO prediction is quite inaccurate 
as it is shown in the experimental validation. The predicted overall desirability by RSMO 
was 0.610 whereas the overall desirability function after the experimental validation was 
0.495. The inaccuracy of the prediction is due to the inaccuracy of the process models, where 
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surface roughness deviation and cutting-tool life models have been fitted by least squares 
with a low coefficient of determination, 89% and 80.4% respectively. 

The methodology proposed in this chapter, based on AI techniques for modeling the 
machining process and the use of search algorithms to optimize the overall desirability 
function on-line, shows an overall desirability function of 0.520. This methodology 
compared with handbook optimization and RSMO increases the overall desirability in 10% 
and 5% respectively. The main benefits reported by this methodology are due to two factors. 
A first factor, the ANN process models let deal with non-linearity so this models are more 
accurate than response surface models for modelling high quality machining operations. 
Unlike RSMO where the error between the predicted overall desirability value and the 
experimental one was 19% (0.61 versus 0.495), the proposed methodology presents an error 
of 5% (0.545 versus 0.520). As a second factor, the on-line nature of the methodology lets 
adapt the cutting parameters every cutting pass so the system is more flexible to adapt any 
change in the objective function during the cutting-tool life. However, the ANN model for 
cutting tool state prediction based on cutting forces seemed to be low accurate due to the 
high variability of the cutting forces during machining. This variation produces that the 
cutting tool parameters selected each cutting tool pass were quite irregular. This effect is 
reflected in Fig. 13b, where cutting speed often varies from 195 to 150-130 m/min. The 
replacement of the direct measurement of cutting forces by indirect methods which are not 
so sensitive to cutting mechanisms such as current or power sensors might increase the 
overall desirability function and select optimal parameters with a more regular variation. 



£»Q,60 



ES 



0,50 



O g 0,40 
Q 

0,30 



f^Ww^ 



12 3 4 5 6 7 8 9 10 11 12 13 141516 1718 192021 2223242526 
Cutting pass 



■a 



200 
_180 
W§160 
.£ 1 140 

3 120 

O 

100 




12 3 4 5 6 7 8 9 101112131415161718192021222324252627 
Cutting pass 



I- ' 12 1 



o i 0,10 

^ £ 0,08 

* 2 0,06 

0,04 




>• • > 



12 3 4 5 6 7 8 9 101112131415161718192021222324252627 
Cutting pass 



Fig. 13. (a) Overall desirability function along cutting tool life-cycle. Parameter optimization 
based on AI. (b) Adaptive cutting speed, (c) Adaptive feed per tooth 
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9. Conclusions 

In this chapter, an Adaptive Control Optimization (ACO) system was presented for 
optimising a multi-objective function based on material removal rate, quality loss function 
related to surface roughness, and cutting-tool life subjected to surface roughness 
specification constraints. Unlike traditional optimization techniques, this methodology lets 
adapt the cutting parameters during the cutting-tool life-cycle in order to maximise in real- 
time the multi-objective function according to cutting-tool state. The core of the system is 
composed of three process models: a cutting- tool wear model for diagnosing the state of the 
cutting tool, a surface roughness deviation model for predicting the quality loss function 
and a cutting-tool life model. All models were developed using Artificial Neural Networks 
(ANN) to model the non-linear relationships in machining processes. The cutting parameter 
optimization is obtained applying genetic algorithms and mesh adaptive direct search 
algorithms. The system proposed was compared with 2 traditional methods for off-line 
cutting parameter selection: (1) selection based on suggested cutting parameters from 
handbooks, and (2) selection based on RSMO. 

The results showed how conservative are the cutting parameters selected by off-line 
methodologies. A cutting parameter optimization based on handbooks provided an overall 
desirability function of 0.472, whereas cutting optimization through RSMO gave an overall 
desirability value of 0.495. The main inconvenient of RSMO is that this methodology is 
based on mathematical first and second order models which are not enough accurate for 
high quality machining operations. The new methodology proposed based on AI techniques 
increases the overall desirability function up to 0.520. The improvement is due to two 
effects: (1) the ANN process models deal with non-linearity so these models are more 
accurate than response surface models for modelling high quality machining operations; (2) 
the on-line nature of the methodology lets adapt the cutting parameters every cutting pass 
so the system is more flexible to adapt any change in the objective function during the 
cutting-tool life-cycle. 
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1. Introduction 

The ultimate objective of any Optical Character Recognition (OCR) system is to simulate the 
human reading capabilities. That is why OCR systems are considered a branch of artificial 
intelligence and a branch of computer vision as well [Sunji Mori (1999)] Character 
recognition has received a lot of attention and success for Latin and Chinese based 
languages, but this is not the case for Arabic and Arabic-like languages such as Urdu, 
Persian, Jawi, Pishtu and others [Abdelmalek Z (2004)]. Researchers classify OCR problem 
into two domains. One deals with the image of the character after it is input to the system 
by, for instant, scanning in which is called Off-line recognition. The other has different input 
way, where the writer writes directly to the system using, for example, light pen as a tool of 
input. This is called On-line recognition. 

The online problem is usually easier than the offline problem since more information is 
available [Liana M & Venu G (2006)]. These two domains (offline & online) can be further 
divided into two areas according to the character itself that is either handwritten or printed 
character. Roughly, the OCR system based on three main stages: preprocessing, feature 
extraction, and discrimination (called also, classifier, or recognition engine) Figure 1.1 
depicts the block diagram of the typical OCR system. Traditional OCR systems are suffering 
from two main problems, one comes from features extraction stage and the other comes 
from classifier (recognition stage). Feature extraction stage is responsible for extracting 
features from the image and passing them as global or local information to the next stage in 
order to help the later taking decision and recognizing the character. Two challenges are 
faced; if feature extractor extracts many features in order to offer enough information for 
classifier, this means many computations as well as more complex algorithms are needed. 
Thus, long processor time will be consumed. On the other hand, if few features are extracted 
in order to speed up the process, insufficient information may be passed to classifier. The 
second main problem that classifier is responsible for, is that most of classifiers are based on 
Artificial Neural Networks (ANNs). However, to improve the intelligence of these ANNs, 
huge iterations, complex computations, and learning algorithms are needed, which also lead 
to consume the processor time. Therefore, if the recognition accuracy is improved, the 
consumed time will increase and vice versa. 
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Figure 1.1 The typical OCR block diagram 

To tackle these problems, a new OCR construction is proposed in this chapter, where neither 
features extractor nor ANN is needed. The proposed construction relies on the image 
compression technique (JPEG). Taking advantages of the compressor, that it compresses the 
image by encoding only the main details and quantizes or truncates the remaining details 
(redundancy) to zero. Then generates a unique vector (code) corresponding to the entire 
image. This vector can be effectively used to recognize the character since it caries the main 
details of the character's image. The importance of the main details is that they are common 
amongst the same character which is written by different writers. 

The principle is illustrated and the algorithm is designed and tested in the related sections. 
In the next section a brief review of the related work is described, Arabic character 
characteristics are illustrated in section 3. In section 4, an overview of JPEG compression is 
shown. In section 5, the concept and the proposed algorithm are presented. The 
experimental results are shown and discussed in section 6. The conclusion of the chapter is 
drawn in section 7. 



2. Review of the related work 

Beside the main goal of any OCR system which is simulating human's reading capability, 
the accuracy and time consuming are very important issues in this aspect. Based on the 
latest survey which is published in May 2006 by Liana M. and Venu G. [Liana(2006)] all 
covered papers have presented their proposals seeking high accuracy and less time. Each 
one has treated the issue from different angle of view. Their work can be classified into three 
main categories: preprocessing problems, features extraction problems, and recognition 
(discrimination) problems. For preprocessing stage, where the image is often converted to a 
more concise representation prior to recognition the most common methods are: A skeleton 
which is a one-pixel thick representation showing the centrelines of the character. 
Skeletonization is also called "thinning" facilitates shape classification and feature detection. 
Many researchers have used skeletonization in their proposed preprocessing stages like for 
instant [S. Mozaffari et al, (2005)], [S. Alma'adeed et al, (2002)], [S. Alma'adeed et al, 
(2004)]. Another method which is known as "contour" is the Freeman chain code of the 
character's border. In this method chain code stores the absolute position of the first pixel 
and the relative positions of successive pixels along the character's border. This methods has 
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been practiced by many like [R. Safabakhsh & P. Adibi (2005)], [L. Souici-Meslati & M. 
Sellami (2004)], [N. Farah et al., (2004)], however, skeletonization suffers from 
mislocalization of feature and ambiguities particular to each thinning algorithm whereas the 
contour approach avoids these problems since no shape information is lost. For the features 
extraction stage, where the main role is played and mostly the accuracy of recognition 
depends on the information passed from this stage to the classifier (recognizer). These 
information can be structural features such as loops, branch-points, endpoints, and dots or 
statistical which includes but is not limited to, pixel densities, histograms of chain code 
directions, moments, and Fourier descriptors. Because of the importance of this stage many 
approaches and techniques have been proposed. For instant, Clocksin in [W.F. Clocksin & 
P.P.J. Fernando (2003)] applied moment functions to image and polar transform image. 
Many like [A. Amin (2003)], [G. Olivier et al, (1996)], [I.S.I. Abuhaiba et al, (1998)], used 
loops, dots, curves, relative locations, height, sizes of parts of characters, loop positions and 
types, line positions and directions, and turning points. Where others used statistics from 
moments of horizontal and vertical projection like[H. Al-Yousefi & S.S. Udpa (1992)]. 
Histogram of slopes along contour is used by [M. Dehghan et al., (2001)]. Fourier descriptors 
have also been used in this stage by [R. Safabakhsh & P. Adibi (2005)]. Few used techniques 
such as wavelet or fractal like [Amir M. et al, (2002)], [Saeed M. et al, (2004 a)] and [Saeed 
M. et al., (2004 b)]. Artificial Neural Networks (ANNs) are the common seed of most if not 
all classifiers "recognition or discrimination stage". Many variations have been used in 
order to overcome the main disadvantage of ANNs which is time consuming. Sherif and 
Mostafa in [Sherif K. & Mostafa M (1996)], for example, presented a parallel design for 
backpropagation Neural Networks approach in order to accelerate the computation process. 
If we want to mention who did use ANNs in OCR, we may list all of them at least within 
our concern field in this chapter "off-line Arabic handwriting character recognition". 

3. Arabic character characteristics 

The Arabic alphabet contains basically 28 letters are written from write to left. Each letter 
can take from two to five different shapes, thus, roughly the alphabet set can expand to 84 
different shapes according to the position of the letter (beginning, middle, end or isolated) as 
well as according to the style of writing (Nasekh, Roqa'a, Farisi and few others). This is one 
reason makes Arabic recognition complex. The second reason is the similarities among the 
different letters and the differences among the same latter. For example, letters Baa, Taa, and 
Thaa (number 2,3 & 4 respectively in Arabic Alphabet) are three different letters, but they 
have similar body shape, they only differ in number and position of dots (one, two or three 
dots below or above the body of the character). Also Jeem, Hhaa & Khaa (number 5, 6, & 7 
respectively in Arabic Alphabet) differ only in one dot. On the other hand the differences 
among the same letter, for example, letter Haa (number 26 in the Arabic Alphabet) has three 
completely different shapes through its positions. Officially, there is a set of rules to write 
Arabic letters, but few follow. These rules may help OCR to extract features or segment text, 
such as so called base-line rule. This rule states that there are three lines. Each letter or group 
of letters has their lines where they should be lie. 

4. JPEG compression background 

The most important current standard for image compression is JPEG [W.B Pennebaker & 
J.L. Mitchell (1993)]. In the JPEG baseline coding system, which is based on the discrete 
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cosine transform (DCT) and is adequate for most compression applications, the input and 
output images are limited to 8 bits, while the quantized DCT coefficient values are 
restricted to 11 bits. The human vision system has some specific limitations, which JPEG 
takes advantage of to achieve high rates of compression. 
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Figure 4.1 JPEG block diagram, (a) encoder and (b) decoder 

As can be seen in the simplified block diagram of Figure 4.1, the compression itself is 
performed in four sequential steps: 8x8 sub-image extraction, DCT computation, 
quantization, and variable-length code assignment. Since we do not concern in this work 
about the reconstruction part, the only part of compression is used (dashed box) and the 
vector will be tapped immediately after quantization stage. 

5. Proposed algorithm 

The main concept of the proposed algorithm is based on the property that the JPEG 
compressed image is a vector which can uniquely represent the input image to be correctly 
reconstructed later at decompression stage. This property can be effectively used to 
recognize the character's image. A JPEG approximation MATLAB code is used to generate a 
vector of coefficients that represent the important information (or main details for the 
character's image). In the next sections the experimental work is illustrated 



5.1 Experiment's tools and platform environment 

The data consists of groups for the twenty eight Arabic alphabets, each group was written 
by 48 writers from different ages and educational backgrounds. Figure 5.1 shows one 
sample of one group 'Tetter Sheen". 

An 40x40 8-bit pixel colour image was used as input image. For experimental reason, the 
MATLAB version 7.2.0.232 (R2006a) has been used but in order to improve the speed of the 
proposed algorithm a prototype of C++ code might be used. The Intel Pentium 4 CPU, 
3.00GHz clock and 960MB RAM of the computer, has been used 
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Figure 5.1 A group sample of letter Sheen 

5.2 Proposed algorithm top-down flowchart 

Figure 5.2 illustrates the sequence of the proposed algorithm's steps. After the character's 
image is scanned in the system the JPEG approximation MATLAB code will produce a 
vector. This vector is assumed to uniquely represent input image since it carries the 
important details of that image. Figure 5.3 shows a sample for letter Sheen and letter Ain. 
Then Euclidean distance between this vector and each vector in codebook will be measured. 
Finally, the minimum distance points to the corresponding character, and then the character 
is recognized. 

5.3 Proposed system components 

The two main components are the code of the compression stage (i.e., JPEG compressor in 
the flowchart shown in Figure 5.2) and the codebook 




Figure 5.2 Flowchart diagram of the proposed algorithm 
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Figure 5.3 Sample from the produced JPEG-approximation vector; (A) for letter Sheen, (B) 
for letter Ain 



5.3.1 Compressor MATLAB code 

The code listed in List 5.1 is to compress an image (x) using a JPEG approximation based on 

8x8 DCT transforms, coefficient quantization, and Huffman symbol coding. Input "Quality" 

determines the amount of information that is lost and compression achieved. Y is an 

encoding structure containing fields: 

[ y.size] is the size of the image (x) 

[y.numblocks] is the number of the 8x8 encoded blocks 

[y.quality] is the quality factor (as percent) 

[y.huffman] is the Huffman encoding structure, as returned by mat2huff (see List 8.1) 

% This function appeared in [R.C. Gonzalez, R. E. Woods, & S.L.Eddins (2003)] 

function y=im2jepg(x,quality) 

global vctr % vctr is the vector obtained after the image is 

% JPEG compreseed and before Huffman takes place 
error(nargchk(l,2,nargin)); % check input argument 
if ndims(x) ~=2 | -isreal(x) | -isnumeric(x) | ~isa(x, , uint8') 

error('the input MUST BE a UINT8 image'); 
end 
if nargin<2 

quality =1; % Default value for quality 

end 
m=[ 16 11 10 16 24 40 51 61 

12 12 14 19 26 58 60 55 

14 13 16 24 40 57 69 56 

14 17 22 29 51 87 80 62 
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18 22 37 56 68 109 103 77 
24 35 55 64 81 104 113 92 
49 64 78 87 103 121 120 101 
72 92 95 98 112 100 103 99] * quality; 
order=[ 1 9 2 3 10 17 25 18 11 4 5 12 19 26 33 ... 

41 34 27 20 13 6 7 14 21 28 35 42 49 57 50 ... 

43 36 29 22 15 8 16 23 30 37 44 51 58 59 52 ... 

45 38 31 24 32 39 46 53 60 61 54 47 40 48 55 ... 

62 63 56 64]; 
[xm,xn]=size(x) 
x=double(x)-128; 
t=dctmtx(8); 
% Compute DCTs of 8x8 blocks and quantize the coefficients 
y=blkproc(x,[8 8], , Pl*x*P2 , ,t,t'); 
y=blkproc(y,[8 8],'round(x./Pl) , ,m); 

y=im 2col(y,[8 8], 'distinct'); % Break 8x8 blocks into columns 
xb=size(y,2); 
y=y(order,:); 

eob=max(x(:))+l % Create end-of-block symbol 

r=zeros(numel(y)+size(y,2),l); 
count=0; 
for j=l:xb 

i=max(find(y(:,j))); 

if isempty(i) 
i=0; 

end 

p=count+l; 

q=p+i; 

r (p : q)=[yOHj); eob ]; 

count=count+i+l; 
end 
count 

r((count+l):end)=[]; % Delete unused portion of r 
y.size=uintl6([xm xn]); 
y.numblocks= uintl6(xb); 
y.quality = uintl 6 (quality *100); 
y.huffman =mat2huff (r); 
vctr=r; % This is the produced vector for the proposed system 

List 5.1 MATLAB code for the compressor stage 

In accordance to the block diagram of Figure 4.1 (A) " dashed box", function im2jpeg 
processes distinct 8x8 sections or blocks of input image x one block at a time (rather than the 
entire image at once). Two specialized block processing functions - blkproc and im2col - 
were used to simplify the computations (more information available @ MATLAB 
Mathworks® documentary). Function blkproc is used to facilitate both DCT computation 
and coefficient denormalization and quantization, while im2col is used to simplify the 
quantized coefficient reordering and zero run detection. The function im2jpeg listed in List 
5.1 uses an alternate matrix formulation 



28 Advances in Robotics, Automation and Control 

T = HFH (1) 

Where F is an 8x8 block of image f (x,y), H is an 8x8 DCT transformation matrix generated by 
dctmtx(8) (more details for MATLAB functions can be found in MATLAB® Documentary), 
and T is the resulting DCT of F. note that the H is used to denote the transpose operation. 
The statement y = blkproc(x,[8 8], 'Pl*x*P2',h / h') computes the DCTs of image x in 8x8 
blocks, using DCT transform matrix h and transpose h' as parameters PI and P2 of the DCT 
matrix multiplication, Pl*x*P2 

5.3.2 Codebook building 

The codebook can be built as following: 

1. Apply the MATLAB code listed in List 5.1 to all available database (our database 
contains 1968 written characters) to get 1968 vectors 

2. Group the 1968 vectors (vctr) according to their represented character. For instance, the 
group of letter Sheen has (in our database) 48 different Sheen that were written by 48 
different writers, so it will have 48 vectors (vctr) 

3. Average each group (MATLAB function "mean" was used in our experiments). By now 
you should get one vector (vctr) for each group. These are the codes (vctrs) located in 
the codebook. 

6. Results and discussion 

Once the codebook is installed the algorithm is ready for testing. Test should cover all 
available data (1968 tests) to ensure that the code book can handle wide range of expected 
character images and recognize them with minor errors. Two criteria may be used to 
examine the system, accuracy and rate of recognition. The accuracy can be measured as the 
percent of the correctly recognized characters to the tested characters. The rate of 
recognition (or the speed) is the time system takes from the image inputs until it is 
recognized. Since the process is a character's shape-independent, the taken times will be 
equal for all images. Adding a timer code can return the time spent by the system. In 
addition to JPEG approximation MATLAB code the system needs to calculate Euclidean 
distance and the minimum value (the shortest distance). The Euclidean distance (d) between 
two vectors X and Y can be defined as : 

d = ^(x-y) 2 (2) 

Whose MATLAB function is 

D=dist(x, y) (3) 

Table 6.1 shows the accuracy recognitions' percentage for each character obtained by the 
proposed algorithm. The overall average is also shown. The system was able to recognize 
the characters during short time comparing with any existing system using ANN because it 
saves time taken by features extractor as well as it uses codebook (lookup table) instead of 
ANN. Misrecognition occurred to some letters is because of the nature of the handwritten 
character itself neither the algorithm. Thus, there is no rejection state, the character must be 
recognized in all situations. It is either correct recognized or incorrect recognized this 
confusing in recognition may occur even to humans. To verify this conclusion the algorithm 
was examined with printed characters and obtained better results. 
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Table 6.1 The recognition accuracy 

Moreover, to test algorithm's intelligence some experiments were done such as taking off 
the dot from letter Khaa £ or Jeem £ the algorithm recognized them as Hhaa £, and from 
letter Dhad <j^ the algorithm recognized it as Sad o». Also from Theal i and Zaay j they 
were recognized as Dal ■> and Raa j respectively. Furthermore, when a character was turned 
upside-down, the farthest Euclidian distance; instead of the shortest Euclidian distance, 
pointed to that character 

7. Conclusion 

In this chapter we proposed a new structure of off line OCR system which is not based on 
ANN, to avoid the time consuming problems. Moreover, it benefited from the JPEG image 
compression property that is high compression ratio which produces minimum compressed 
image size and every compressed image has a unique vector which helps to identify each 
character. By using this unique vector, the proposed system has recognized the input 
character after measuring the Euclidean distance between the vector and the vectors in the 
codebook, then the shortest distance pointed to the corresponding letter. In addition to the 
advantage of speed using codebook, it can be universal by means of character's nature 
(language, writing mode) as well as character's image size. We used 40x40 8-pexil color 
image as input image. The result was considerably high in terms of accuracy and 
recognition rate. We used JPEG approximation MATLAB code . Future work could be done 
using in advanced on line OCR system, also could be implemented by using fast 
programming languages such as C++ and it is highly recommended to be used for different 
Biometrics applications. 



8. Appendix: MATLAB code 

% This function appeared in [R.C. Gonzalez, R. E. Woods, & S.L.Eddins (2003)] 
function y=mat2huff (x) 
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if ndims(x)~= 2 | -isreal(x) | (-isnumeric(x) & -islogical(x)) 

error('X must be a 2-d real numeric or logical matrix'); 
end 

% store the size of input x 
y.size = uint32(size(x)); 

% find the range of x values and store its minimum value biased 
% by +32768 as a UINT16 
x=round (double (x) ); 
xmin=min(x(:)); 
xmax=max(x(:)); 
pmin= double (intl 6 (xmin) ) ; 
pmin= uintl6(pmin+32768); y.min=pmin; 

% compute the input histogram between xmin and xmax with uint 
% width bins, scale to UINT16, and store 
x=x(:)'; 

h=histc(x,xmin:xmax); 
if max(h) > 65535 

h=65535*h/max(h); 
end 

h=uintl6(h); y.hist=h; 

%code the input matrix and store the result, 
map = huffman(double(h)); % Make a Huffman code map 
hx=map(x(:)-xmin+l); % Map image 

hx=char(hx)'; % Convert to char array 

hx=hx(:)'; 

hx(hx==' ')=[]; % Remove the blanks 

ysize=ceil(length(hx)/16); % Compute encoded size 
hxl6=repmat('0 , ,l,ysize*16); % Pre-allocate modulo-16 vector 
hxl6(l:length(hx))=hx; % Make hx modulo-16 in length 

hxl6=reshape(hxl 6,1 6,y size); % Reshape to 16-character words 
hxl6=hxl6 , - , 0'; % Convert binary string to decimal 

twos=pow2(15:-l :0); 
y.code=uintl6(sum(hxl6.*twos(ones(ysize,l),:),2))'; 

List 8.1 MATLAB code (mat2huff) 
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1. Introduction 

High rates of manufactured items have been machined by grinding at some stage of their 
production process, or have been processed by machines whose precision is a direct result of 
abrasive operations. However, even being the grinding process the most used in industry 
for obtaining high level of surface quality, it remains as one of the most difficult and least 
understood processes (Wang et al., 2005). That maybe has origin in the mistaken faith the 
process is extremely complex to be understood due to the large number of cutting edges and 
irregular geometry, high cutting speed, and very small depth of cut which varies from grain 
to grain. In addition, according to (Haussi & Diniz, 2003), grinding is the process indicated 
when the workpiece demands good surface, dimensional and geometrical quality. Thus, the 
grinding process is usually one of the last steps in the machining operations chain. When the 
workpiece reaches this point, it has high aggregated value, which makes a possible rejection 
very expensive. 

Monitoring of machining processes is mandatory for their optimization and control. 
Acoustic emission (AE) has become an increasingly popular monitoring technique. The 
sensors are inexpensive, easy to mount, and analog signal processing is comparatively 
simple, but good techniques for extracting reliable process information from the signals are 
still lacking (Hundt et al, 1997; Aguiar et al, 2002). 

Electrical power signals have also been largely used in grinding researches. The signal can 
be monitored either by the electric current of the electric motor or by the product between 
voltage and current signals, which gives the electrical power consumed by the electric 
motor. Thus, an estimate of the cutting force can be easily obtained if a model of the electric 
motor is available (Aguiar et al., 2002). 

Some researchers have shown the acoustic emission and the cutting power signals combined 
can provide significant results for monitoring the grinding process phenomena (Aguiar et 
al, 2002; Dotto et al, 2006; Kwak & Ha, 2004; Aguiar et al, 2006). 

Neural network has attracted a special interest in grinding research owing to its functions of 
learning, interpolation, pattern recognition, and pattern classification. Various examples of 
applications into the production engineering field have been reported (Wang et al., 2005; 
Dotto et al, 2006; Kwak & Ha, 2004; Aguiar et al, 2006; Wang et al, 2001). 
According to (Wang et al., 2005), surface roughness is one of the most important factors in 
assessing and determining the quality of a part. In practical, predicting and controlling the 
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roughness is difficult due to the fact that many variables are affecting the process and many 
of these variables are non-linear, interdependent, or difficult to quantify with crisp numeric 
precision. Still, taking into account that some grinding processes can only be represented by 
experimental data or linguistic descriptions, the usage of intelligent systems into the 
optimization of grinding processes appears to be inevitable. 

The objective of this work was to use neural networks to predict the surface roughness of 
ground workpieces based on the analysis of several process output variables, such as 
acoustic emission, cutting power, and other statistics generated from these signals. 

2. Literature review 

Machining with grinding wheels is a very complex process affected by so many factors that 
a reproducible result is rarely obtained. The most important one is that the cutting ability of 
the grinding wheel changes considerably during the grinding time. In practice, the grinding 
process is carried out with cutting parameters which are safe but not optimal. The work- 
piece quality depends to a great extent on the experience of the operator (Lezanski & 
Rafalowicz, 1993). 

The result of a grinding process can be subdivided into characteristics concerning the 
geometry and surface integrity of a ground component. The geometrical quantities are 
dimension, shape and waviness, as essential macro-geometric quantities; whereas the 
roughness condition is the main micro-geometric quantity. The surface integrity state can be 
described by residual stresses, hardness and structure of the material (Brinksmeier et al., 
1998). 

According to (Bhushan, 2001), solid surfaces, irrespective of their method of formation, 
contain irregularities or deviations from the prescribed geometrical form. The surfaces 
contain irregularities of various orders ranging from shape deviations to irregularities of the 
order of interatomic distances. No machining method, however precise, can produce a 
molecular ly flat surface on conventional materials. Even the smoothest surfaces, such as 
those obtained by cleavage of some crystals, contain irregularities, the heights of which 
exceed the interatomic distances. For technological applications, both macro and micro- 
nanotopography of the surfaces (surface texture) are important. 

A very general topology of a solid surface is seen in Figure 1. Surface textures that are 
deterministic may be studied by relatively simple analytical and empirical methods; their 
detailed characterization is straightforward. However, the textures of most engineering 
surfaces are random, either isotropic or anisotropic, and either Gaussian or non-Gaussian. 
Whether the surface height distribution is isotropic or anisotropic and Gaussian or non- 
Gaussian depends upon the nature of the processing method. Surfaces that are formed by 
cumulative processes (such as peening, electropolishing, and lapping), in which the final 
shape of each region is the cumulative result of a large number of random discrete local 
events and irrespective of the distribution governing each individual event, will produce a 
cumulative effect that is governed by the Gaussian form. It is a direct consequence of the 
central limit theorem of statistical theory. Single-point processes (such as turning and 
shaping) and extreme-value processes (such as grinding and milling) generally lead to 
anisotropic and non-Gaussian surfaces. The Gaussian (normal) distribution has become one 
of the mainstays of surface classification (Bhushan, 2001). 

A typical parameter that has been used to quantify the quality of a surface topography is the 
surface roughness, which is represented by the arithmetic mean value, Ra, the root-mean- 
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square-average, Rq, and the maximum roughness height, Rt. In general, the longitudinal 
surface roughness has a lower value than the traverse surface roughness; therefore the latter 
is more frequently used in industry (Hecker & Liang, 2003). Still, according to (Kwak et al, 
2006) the surface roughness obtained in grinding depends in a complex way upon the 
roughness of the wheel surface, the grinding parameters, and tribological interactions 
between the abrasive cutting points and the workpiece. 



Solid surface 



Inhomogeneous 
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Homogeneous 
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Gaussian 
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Fig. 1. General typology of surfaces (Bhushan, 2001) 

Several sensing schemes for monitoring tool condition and cutting status in normal 
machining have been proposed and evaluated in the last two decades. However, the 
technique which involves monitoring the acoustic emission generated during the machining 
operation has been found to be very sensitive to elements of metal removal, such as sliding 
contact, plastic deformation, phase transformation, micro-cracking, fracture, impacts and so 
on (Dornfeld, 1985). The acoustic emission sensing technique uses the information contained 
in the transient elastic stress wave, which is generated by rapid release of energy within a 
material, to provide knowledge about the state of the process. 

The application of signals from the cutting process became a more interesting tool when 
neural networks are used for their processing and interpretation. This tool has attracted 
interest of several researchers in the surface roughness prediction (Wang, 2005; Aguiar et al., 
2006; Kwak et al, 2006; Fredj et al, 2002) . 

Artificial neural networks have been studied for many years in the hope of achieving the 
human-like performance in the field of the speech, image recognition and the pattern 
classification. These neural networks are composed of many non-linear computational 
elements operating in parallel. Neural Networks, because of their massive nature, can 
perform computations at a higher rate. Because of their adaptative nature using the learning 
process, neural networks can adapt to changes in the data and learn the characteristics of the 
input signals (Kwak & Ha, 2004). Still, according to the author, the ability to learn is a 
fundamental trait of the neural network. Although a precise definition of learning is difficult 
to formulate, the learning in a neural network means the finding an appropriate set of the 
weights that are connection strengths from the elements to the other layer elements. 
What makes this work distinguished from others is the use of grinding parameters as input 
to the neural network, which have not been tested yet in surface roughness prediction by 
neural networks. Besides, a high sampling rate data acquisition system was employed to 
acquire the acoustic emission and cutting power. 
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3. Methodology 

The materials, methods and equipments used for the development of the grinding tests, 
surface roughness measurements, training and validation of the neural networks will be 
presented in the next section. 

3.1 Experimental set-up and grinding parameters 

The workpieces for the grinding tests consisted of laminated bars of steel SAE 1020 ground 
in the shape of a prism with 150mm length, 12.7mm width and 43mm height. A surface 
grinding machine from Sulmecanica manufacturer, Brazil, model RAPH-1055 was used in 
the grinding tests. The grinder was equipped with an aluminium oxide grinding wheel, 
from Norton Manufacturer, Model ART-FE-38A80PVH. 



Frequency Inverter 



Computer 




Figure 2. Schematic diagram of the grinding machine and instrumentation 

A fixed acoustic emission sensor from Sensis manufacturer, model DM-42, placed near the 
workpiece and an electrical power transducer for measuring the electrical power consumed 
from the three-phase induction motor that drives the wheel were employed. The power 
transducer consists of a Hall sensor to measure the electric current and a Hall voltage sensor 
to measure the voltage at the electric motor terminals. Both signals are processed internally 
in the power transducer module by an integrated circuit, which delivers a voltage signal 
proportional to the electrical power consumed by the electric motor. The acoustic emission 
as well as the power signal are further sent to the data acquisition board from National 
Instrument, model PCI-6011, which is installed into a personal computer. 
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The Lab VIEW software was utilized for acquiring the signals and storing them into binary 
files for further processing and analysis. The acoustic emission sensor used has a broad- 
band sensitivity of 1.0 MHz. Its amplifier also filtered the signal outside the range of 50 kHz 
to 1.0 MHz. Figure 2 shows the schematic diagram of the grinding machine and 
instrumentation used. 

The tests were carried out for 15 different grinding conditions, using 5um as the lowest 
depth of cut and 50 |um as the largest one. Dressing parameters, lubrication and peripheral 
wheel speed were adequately controlled in order to ensure the same grinding condition for 
the three repetitions of each test. The workpiece speed was set up at 0.043 m/s and the 
wheel speed at 30 m/s. The latter was maintained constant by adjusting the frequency of the 
induction motor on the frequency inverter, as the grinding wheel had its diameter decreased 
along the tests. 

The dressing overlap ratio (Ud), which determines the wheel sharp level (Malkin, 1989), was 
set to 1 for all the tests. To control this parameter, the width of the dressing diamond tip was 
measured before each test. The measurements have been done using a profile projector 
which allows micrometer precision. Figure 3 shows the scheme used to measure the 
dressing diamond tip. 




Figure 3. Method used to measure dressing tip width (bd) on a profile projector. 
Using equation 1 is possible to determine the dressing speed 



Vj= nb d 



U d .60 



(1) 



Where: Vd is the dressing speed; L r is the wheel width (31,75 mm); n the wheel rotation 
(1800rpm). For a better understanding, figure 4 was built to show some details on the 
dressing operation used in this work. It can be observed the single-point diamond moving 
across the wheel surface with direction given by the speed vector va. 

A water-based fluid was used with 4% concentration. Each run consisted of a single 
grinding pass of the grinding wheel along the workpiece at a given grinding condition to be 
analyzed. The acoustic emission and cutting power signals were measured in real time at 2.5 
millions of samples per second rate, and then stored onto binary data files for further 
processing. It is important to mention that the raw acoustic emission signal was acquired 
instead of the root mean square generally used. 
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dressing diamond tip 




Figure 4. Dressing operation with single-point diamond. 

3.2 Surface roughness measurements 

Once all the grinding tests were done, surface roughness measurements were taken using a 
Surtronic 3+ tester and Taylor Hobson's Taly Profile software, version Lite 3.1.4, adjusted to 
a length of 8 mm sampling rate. Figure 5 shows the regions along the workpiece where the 
surface roughnesses were taken which accounted for 15 measurements. The surface 
roughness Ra was defined as the measurement parameter. 
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Figure 5. Lines showing the 15 regions along the workpiece where the surface roughness 
was measured 

These data on surface roughness were naturally used in the neural networks, and they allow 
to verifying the magnitudes and efficiency of the system for prediction. 



3.3 Statistical parameters used in the neural networks 

Many parameters for monitoring fault in the grinding process have been studied. Two 
important parameters investigated by (Aguiar et al., 2002) and (Dotto et al., 2006) are DPO 
and DPKS. The DPO is defined as the standard deviation of the root mean square of acoustic 
emission signal multiplied by the maximum value of electric power in the grinding cycle or 
pass. The DPKS is the sum of the difference to the fourth power between cutting power and 
standard deviation of the cutting power multiplied by the standard deviation of the root 
mean square acoustic emission in the grinding cycle or pass. The DPO and DPKS are shown 
by equations 2 and 3 respectively. 
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DPO = std(AE)Max (Power) (2) 

DPKS = {£ [Powerii) - std (Power)] 4 }std(AE) (3) 

i=\ 

Where Max is the maximum value; Power the cutting power; AE the acoustic emission; std 
the standard deviation, m was set to 1024 in this work. 

Thus, DPO and DPKS parameters were obtained from the data files stored during the 
acquisition. As these parameters were efficient at detecting grinding burn, they were tested 
as being the inputs of the neural networks in this research. 

3.4 Neural network and input sets 

Among several well known techniques of artificial intelligence are artificial neural networks, 
which basically consist of computational models analogous to the human brain whose main 
characteristic is the capability of learning (Aguiar et al, 2006). They are composed of many 
non-linear computational elements operating in parallel fashion. Neural networks, because 
of their massive nature, can perform computations at a higher rate. Due to their adaptive 
nature in using the learning process, neural networks can adapt to changes in the data and 
learn the characteristics of input signals (Kwak & Ha, 2004). In this work, the back- 
propagation algorithm of neural networks, which is one of the learning models, was used. 
Three hidden layers composed of 60, 40 and 20 neurons were found to be the best 
configuration for the neural networks studied. The following parameters were also found 
more suitable: sigmoid tangent activation function for the neurons of the hidden layers; 
linear activation function for the neurons of the last layer, downward gradient training 
algorithm; all data in the neural networks were normalized; training for 10000 epochs; 
square mean error value of 10-5; learning rate of 0.3; and momentum coefficient of 0.4. Table 
1 shows three different sets used for the neural networks. 



Set 


Neural Network Configuration 


1 


AE, Cutting Power and Depth of Cut 


2 


AE, Cutting Power, DPO, DPKS and Depth of Cut 


3 


DPO and Depth of Cut 



Table 1. Neural Network Configurations 

4. Results and discussion 

The measurements of surface roughness have shown an increase in magnitude as the depth 
of cut was increasing, mainly after the test with 35 jum depth of cut where grinding burn on 
the workpiece surface took place. In order to illustrate that increase in surface roughness, 
Figure 6 shows the mean values of surface roughness (|im) calculated for each depth of cut 
used for 15 regions along the workpiece (Figure 5) and three repetitions for each grinding 
condition. It can be observed the high standard deviation values more easily visible to those 
conditions of greater depth of cut, due mostly to encompass the mean of surface roughness 
along the workpiece. This factor can be better explained by the difference of surface 
roughness along the workpiece length; mainly in the tests where in certain region of the 
workpiece burn occurred. Thus, there was a great difference between the surface roughness 
values for the region where burn took place and the region with no burn. 
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Figure 6. Average of Surface Roughness Ra (|um) along the workpiece in function of the 
depth of cut (|um) for 15 grinding runs. 

It is important to point out that Figure 6 was built just for easily visualization of the surface 
roughness behaviour in relation to the depth of cut, provided the training and validation 
tests in the neural networks the surface roughness for each measured region was used. 
From the grinding tests acoustic emission and cutting power vectors were extracted. 
Following each test, the root mean square of AE and filtering of both signals were obtained 
in order to check the success of the test as shown in Figure 7. 





Figure 7. Acoustic Emission (a) and Cutting Power (b). Horizontal axis in seconds; Vertical 
axis in Volts. 

These vectors were applied to the training of the neural networks, which provided surface 
roughness values and allowed the analysis of the system efficiency. The quantification of 
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errors was done point by point through the computation of the absolute value of the 
difference between the measured values of surface roughness and the values given by the 
neural network. That was done for each of the 15 regions along the workpiece and for each 
workpiece. 

Figure 8 shows the average values of square errors for all 15 regions along the workpiece 
length and 3 repetitions of each grinding condition as well as the variation of the error in 
each of those conditions. The lateral bar exhibits the three groups of errors found, that is, the 
lowest one the success, errors about 10 1 um in the middle in green, and errors equal to or 
greater than 1 um on the top in red. 
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Figure 8. Average values of square errors, maximum and minimum errors values for each of 
15 workpiece regions and 3 repetitions of each grinding condition. 

It can be seen in Figure 8 the set number 1 presented error quite high to those greater depths 
of cut studied, overreaching the value of 1 |um for the last three grinding conditions. As 
mentioned previously, from the analysis of workpiece integrity it can be observed the 
grinding burn started at depth of cut of 35 |um. Thus, acoustic emission, cutting power and 
depth of cut have not responded very well as inputs to the neural network in the surface 
roughness prediction. However, the error observed does not necessarily mean that those 
parameters are not adequate to monitor other grinding control parameters, for instance, the 
grinding burn. The set number 2 has also presented greater errors in the last three grinding 
conditions but they have not overreached in average value of 1 |um. 

Based on this information, the difference between the real measurements and the output of 
the neural networks can be classified in three groups: The first group for errors not greater 
than 10 2 |um, the second group for errors of about 10 1 |im, and the third group for errors 
equal to or greater than 1 urn. 

As the surface roughness measurement device used in this work allows measurements with 
precision of two decimal digits, the last digit of this equipment is concerned with the 
uncertainty of the measurement. Thus, all the errors in the range of 10 2 u,m or inferior were 
considered to be successful in the prediction of the surface roughness. This remark is 
reinforced by the tolerances applied to the grinding process in industry, which in general 
has tolerances in the range of 10 1 um. 
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In the second group, errors of about 10 1 \im was thought as bad ones, but it would be 

mostly fine in industry depending on the type of part ground. Finally, the third group of 

errors is in the range of 1 \xm or superior, which is classified as very bad ones. 

Based on these analyses, we can see in Figure 8 that for many conditions, neural network 

had a hundred percent of success or almost that, mainly in the region from 5 ^m of depth of 

cut to 30 jim, proving a total efficiency of the system for certain processes. These conditions 

are, for example, 10 jum, 12.5 |um, 15 jum and 17.5 jum for Set 1 and Set 2, and from 10 jum to 

30 |im for Set 3. 

It is important to point out that the great efficiency of the process occurred in the area of 

industrial application, since the largest rate of success was obtained exactly when burn on 

the workpiece doesn't succeed. 

Comparing the values of surface roughness obtained in the tests and shown in Figure 6 it 

can be inferred that the studied process is optimized using 27.5|um depth of cut. This depth 

of cut represents the largest material removal rate employed before grinding burn takes 

place. Surface roughness values are similar to the grinding conditions before 27.5|um depth 

of cut. 

Figure 9 was built in order to facilitate the comparison among the 3 input Sets and to show 

the general result of the values in all tested conditions. 
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Figure 9. Percentage of errors and success of the neural network prediction for each set 

Observing Figure 9 and other results from the neural network output as well as the 
comparison between the surface roughness values, the following remarks can be summarized. 
From the sets utilized in the neural network training, the set number 3 (DPO and depth of 
cut) has presented the best output values for almost every grinding condition tested. This 
set of inputs has provided about 70% of success in the prediction of surface roughness 
analyzed region by region. In addition, it has not presented any error classified as bad one 
for the grinding process control. 

The set number 1, composed by acoustic emission, cutting power and depth of cut, has 
presented satisfactory performance in the vast majority of the grinding condition tested, 
showing however pretty high errors mainly in the conditions of high depth of cut. Burning 
of the workpieces has occurred in these conditions, which can be understood as if this set is 
sensitive to others parameters besides surface roughness. 
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In the same way as occurred with set number 1, set number 2 (AE, cutting power, DPO, 
DPKS and depth of cut) also had good performance. The values of prediction for these two 
sets have alternated themselves, either for better or for worse when they are compared to 
each other. It is important to highlight that the set number 1 has showed higher errors than 
those seen in the set number 3 for some conditions. However, it has obtained good 
performance in the analysis of bad errors, showing only 3.1% for this kind of error. 

5. Conclusions 

The technique of surface roughness prediction has been developed using multi-sensor 
method with AE sensor and power meter for grinding process. Based on the results 
presented in this research, it can be concluded that acoustic emission and cutting power 
signals are very good input parameters to the neural network for surface roughness 
prediction of ground parts. Thus, the implementation of these signals is feasible in the 
control of grinding process in industry, provided that very low percentage of "bad" errors 
presented by the neural network was found. From the sets used in the training of the neural 
networks, and for the grinding conditions and methods employed, the set number 3 
composed only by the DPO parameter and depth of cut has presented the best results. This 
set has reached about 70% of success in the prediction and none "bad" error, and for the 
condition where workpiece burn didn't occur success at prediction was almost 100%. In the 
comparison of errors generated by each set, set 3 was followed either by the set 1 or set 2. 
Because the level of errors found in the utilization of set number 3 as input to the neural 
network, the use of DPO parameter is highly attractive for predicting surface roughness in 
grinding. Therefore, an analog DPO output signal could be generated by an electronic 
circuitry with no need for further digital processing of AE and cutting power signals. 
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1. Introduction 

Nowadays production flows are modular, each module in the enterprise being specialized and 
used to achieve a particular task. In many cases the modules are interconnected and materials 
are sequentially processed in each module resulting a final, unique product or assembly. One 
typical such production module is a flexible cell/ system using multiple robots. In such 
complex enterprise environments, providing continuous services for applications is a key 
component of a successful implementing of robotized manufacturing. High availability (HA) is 
one of the components contributing to continuous service provision for applications, by 
masking or eliminating both planned and unplanned downtime of systems and applications. 
This is achieved by eliminating hardware and software single points of failure (SPOF). 
The systems configured for high availability are a combination of hardware and software 
components configured to work together to ensure automated recovery in case of failure 
with a minimal acceptable downtime. 

A high availability solution will ensure that the failure of any component of the solution - 
either hardware, software or system management, will not cause the application and its data 
to become permanently unavailable. High availability solutions should eliminate single 
points of failure through appropriate design, planning, hardware selection, software 
configuring, application control, carefully environment control and change management 
discipline. 

In short, one can define high availability as the process of ensuring an application is available 
for use by duplicating and/ or sharing hardware resources managed by a specialized software 
component. A high availability solution in robotized manufacturing provides automated 
failure detection, diagnosis, application recovery, and node (robot controller) re integration. 
The chapter discusses the implementing of a high availability solution in a robotized 
manufacturing structure (cell, line). The solution is based on a High Availability Linux 
Cluster which is responsible for data availability and preservation on a NFS (Network File 
System) file system, and a second cluster - Fabrication Cluster (FC) which runs on Adept 
robot controllers, developed under the V+ programming language. 

2. High availability versus fault tolerance 

Based on the response time and response action to system detected failures, clusters and 
systems can be generally classified as: 
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• Fault-tolerant 

• High availability 

2.1 Fault-tolerant systems 

The systems provided with fault tolerance are designed to operate virtually without 

interruption, regardless of the failure that may occur (except perhaps for a complete site 

going down due to a natural disaster). In such systems all components are at least 

duplicated for both software and hardware. 

This means that all components, CPUs, memory, Ethernet cards, serial lines and disks have 

a special design and provide continuous service, even if one sub-component fails. Only 

special software solutions will run on fault tolerant hardware. 

Such systems are very expensive and extremely specialized. Implementing a fault tolerant 

solution requires a lot of effort and a high degree of customization for all system 

components. 

For environments where no downtime is acceptable (life critical systems), fault-tolerant 

equipment and solutions are required. 

2.2 High availability systems 

The systems configured for high availability are a combination of hardware and software 

components configured to work together to ensure automated recovery in case of failure 

with a minimal acceptable downtime. 

In such industrial systems, the software involved detects problems in the robotized 

environment (production line, flexible manufacturing cell), and manages application 

survivability by restarting it on the same or on another available robot controller. 

Thus, it is very important to eliminate all single points of failure in the manufacturing 

environment. For example, if a robot controller has only one network interface (connection), 

a second network interface (connection) should be provided in the same node to take over in 

case the primary interface providing the service fails. 

Another important issue is to protect the data by mirroring and placing it on shared disk 

areas accessible from any machine in the cluster, directly or using the local area network. 

3. High availability terms and concepts 

For the purpose of designing and implementing a high-availability solution for networked 
robotic stations integrated in a manufacturing environment, the following terminology and 
concepts are introduced: 

RMC: The Resource Monitoring and Control (RMC) is a function giving one the ability to 
monitor the state of system resources and respond when predefined thresholds are crossed, 
so that many routine tasks can be automatically performed. 

Cluster: Loosely-coupled collection of independent systems (nodes - in this case robot 
controllers) organized in a network for the purpose of sharing resources and communicating 
with each other. A cluster defines relationships among cooperating systems, where peer 
cluster nodes provide the services offered by a cluster node should that node be unable to 
do so. These individual nodes are together responsible for maintaining the functionality of 
one or more applications in case of a failure of any cluster component. 
There are two types of high availability clusters: 



An Implementation of High Availability in Networked Robotic Systems 



47 



• Peer domain 

• Managed domain 

The general difference between these types of clusters is the relationship between the nodes. 
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Fig. 1. Peer domain cluster topology. 

In a peer domain (Fig. 1.), all nodes are considered equal and any node can monitor and 
control (or be monitored and controlled) by any other node (Harris et. al., 2004). 
In a management domain (Fig. 2), a management node is aware of all nodes it is managing and 
all managed nodes are aware of their management server, but the nodes themselves know 
nothing about each other. 
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Fig. 2. Managed domain cluster topology. 

Node: A robot controller that is defined as part of a cluster. Each node has a collection of 

resources (disks, file systems, IP addresses, and applications) that can be transferred to 

another node in the cluster in case the node or a component fails. 

Clients: A client is a system that can access the application running on the cluster nodes 

over a local area network. Clients run a client application that connects to the server (node) 

where the application runs. 

Topology: Contains basic cluster components nodes, networks, communication interfaces, 

communication devices, and communication adapters. 

Resources: Logical components or entities that are being made highly available (for 

example, file systems, raw devices, service IP labels, and applications) by being moved from 

one node to another. All the resources that together form a highly available application or 

service are grouped in one resource group (RG). The cluster keeps the RG highly available 
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as a single entity that can be moved from node to node in the event of a component or node 

failure. Resource groups can be available from a single node or, in the case of concurrent 

applications, available simultaneously from multiple nodes. A cluster may host more than 

one resource group, thus allowing for efficient use of the cluster nodes. 

Service IP label: A label matching to a service IP address and which is used for 

communications between clients and the node. A service IP label is part of a resource group, 

which means that the cluster will monitor it and keep it highly available. 

IP address takeover: The process whereby an IP address is moved from one adapter to 

another adapter on the same logical network. 

Resource takeover: This is the operation of transferring resources between nodes inside the 

cluster. If one component or node fails due to a hardware or operating system problem, its 

resource groups will be moved to the another node. 

Failover: Represents the movement of a resource group from one active node to another 

node (backup node) in response to a failure on that active node. 

Fallback: Represents the movement of a resource group back from the backup node to the 

previous node, when it becomes available. This movement is typically in response to the 

reintegration of the previously failed node. 

Heartbeat packet: A packet sent between communication interfaces in the cluster, used to 

monitor the state of the cluster components - nodes, networks, adapters. 

RSCT processes: (Reliable Scalable Cluster Technology). They consist of two processes 

(topology and group services) that monitor the state of the cluster and each node. The 

cluster manager receives event information generated by these processes and takes 

corresponding (response) actions in case of failure(s). 

Group Leader (GL): The node with the highest IP as defined in one of the cluster networks 

(the first network available), that acts as the central repository for all topology and group 

data coming from the RSCT daemons concerning the state of the cluster. 

Group leader backup: This is the node with the next highest IP address on the same 

arbitrarily chosen network, acting as a backup for the Group Leader; it will take over in the 

event that the Group Leader leaves the cluster. 

Mayor: A node chosen by the RSCT Group Leader (the node with the next highest IP 

address after the GL Backup), if such exists; otherwise the Mayor node is the GL Backup 

itself. It is the Mayor's responsibility to inform other nodes of any changes in the cluster as 

determined by the Group Leader (GL). 

Quorum: The notion of quorum is used to ensure that in case of loss of connectivity between 

two subsets of the peer domain only one subset considers itself as the peer domain. The 

quorum is defined as n/ 2+1, where n is the number of nodes defined in the peer domain. A 

peer domain cannot be turned online if less than the quorum of nodes can communicate. 

SPOF: A single point of failure (SPOF) is any individual component integrated in a cluster 

which, in case of failure, renders the application unavailable for end users. Good design will 

remove single points of failure in the cluster - nodes, storage, networks. The implementation 

described here manages such single points of failure, as well as the resources required by 

the application. 

4. RMC architecture and components design 

The most important unit of a high availability cluster is the Resource Monitoring and 
Control (RMC) function, which monitors resources (selected by the user in accordance with 
the application) and performs actions in response to a defined condition. 
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The design of RMC architecture is presented for a multiple-resource production control 
system. The set of resources is represented by the command, control, communication, and 
operational components of networked robot controllers (station controllers) and robot 
terminals (station computers) integrated in the manufacturing cell. 

The RMC subsystem to be defined is a generic cluster component that provides a scalable 
and reliable backbone to its clients with an interface to resources. 
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Fig. 3. The structure of the RMC subsystem. 

The RMC has no knowledge of resource implementation, characteristics or features. The 
RMC subsystem therefore delegates to resource managers the actual execution of the actions 
the clients ask to perform. The architecture allows the following functions to be supported 
on clusters: 

• Provide single monitoring and management infrastructure for clusters. 

• Provide global access to subsystems and resources throughout the cluster. 

• Support operations for configuring, monitoring, and controlling all cluster resources by 
RMC clients. 

• Encapsulate all resource dependent operations. 

• Provide a common access control mechanism across all resources. 

• Support integration with other subsystems to achieve the highest levels of availability. 
To support these functions, the structure of the RSCT Resource Monitoring and Control is 
not simple (see Fig. 3). 

The RMC subsystem and RMC clients need not be in the same node; RMC provides a 
distributed service to its clients. The RMC clients can connect to the RMC process either 
locally or remotely using the RMC API i.e. Resource Monitoring and Control Application 
user Interface (Matsubara et ah, 2002). 

Similarly, the RMC subsystem interacting with Resource Managers need not be in the same 
node. If they are on different nodes, the RMC subsystem will interact with local RMC 
subsystems located on the same node as the resource managers; then the local RMC process 
will forward the request between them. 



50 



Advances in Robotics, Automation and Control 



Each resource manager is instantiated as one process. To avoid the multiplication of 
processes, a resource manager can handle several resource classes. 

The commands of the Command Line Interface are V+ programs (V+ is the robot 
programming environment); the end-user can check and use them as samples for writing his 
own commands. 

4.1 The Command Line Interface (CLI) 

A RMC command line client can access all the resources within a cluster locally (A) and 
remotely (B) located (Fig. 4). The RMC command line interface (Fig. 5) is comprised of more 
than 50 commands (V+ programs): some components, such as the Audit resource manager, 
have only two commands, while others, such as Event Response resource manager, have 15 
commands. 

Node 1 Node 2 



RMC client 
(CLI) 



A K B 
/RMC subsystem 

a5 ... 





RMC subsystem 



[ 1 Resource and Resourse classes 
Fig. 4. The relationship between RMC Clients (CLI) and RMC subsystems. 
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Most RMC commands provide the user with a similar perception. Many commands have 
optional arguments to instruct the realization of the same functions. In addition to optional 
command arguments, RMC defines two environment variables: 

• CT_CONTACT - define the communication target 

• CT_MANAGEMENT_SCOPE - define the scope 

These variables instruct most RMC commands to define their target and scope upon 
retrieving and setting values among cluster nodes. 

Finally, RMC embeds the concept of Resource Data Input, which is a standardized file format 
for holding resource properties. Many RMC commands have a -f option that allows the end 
user to pass arguments to the command through a file instead of typing them on the 
command line. These files have the same Resource Data Input format for all commands so 
that the same file can be used for different commands. Two commands (lsrsrcdef - list 
resource definitions, and lsactdef - list actions definitions) can generate such a file so their 
output can be used as input for other commands. 

4.2 The web-based Graphical User Interface (GUI) 

The RMC graphical user interface is provided by the Web-based System Manager, and allow 
the user to configure the fabrication cluster (a HA cluster composed by industrial robots), 
and also acts as a teleoperation system. 
The system allowing access to the web interface is composed by the following applications 

(Fig- 6): 

The Server Application (SA): Remote visual control and monitoring of multiple robot 

controllers from mobile and stationary matrix cameras. 

• Visual control: the Server Application supports almost all V+ and AdeptVision program 
instructions and monitor commands. Robot training and control are interactive - menu- 
driven and acknowledged by image display in a VISION window. Some of the main 
functions available in this window are: choice of the physical camera and virtual 
cameras (multiple information counterparts of a single physical camera) and of the 
image buffers; selecting the display mode and resolution; histogram and average curve 
contrast analysis; selection of switches and parameters for virtual camera construction; 
display of vision system status; training and planning multiple ObjectFinder models for 
recognition and locating (Automated Visual Inspection - AVI & Guiding Vision for 
Robots - GVR); learning fingerprint models for collision-free grasping; editing, saving, 
loading and running V+ programs. 

• Monitoring: a Monitoring/ Treatment scheme can be defined for each Client/ Station (the 
latter can be selected from a drop-down list of robot controllers connected to the server, 
by adding/ removing them from the Client window). For each client a list of events and 
controller variables to be monitored according to a user-definable timing and 
precedence, and reacted at by user-definable actions/ sequences can be specified in an 
Automatic Treatment Window. 

• Communication management: the Server Application manages the communication with 
the robot controllers and the observation cameras, transfers real-time images from the 
cameras observing the robot workplace and production environment, reports status 
information, stores in a database and displays images taken by the robot camera via its 
controller. Finally, the SA outputs commands which are received from the eClients or 
acknowledges task execution. 
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Fig. 6. The structure of the system which provides the web based interface. 

The eClients Applications (eCA): Java applications running in web browsers. They provide 
portal services and the connection of networked production agents: image data and RV 
program / report management; real-time robot control and cell / workplace observation. 
The eCA are composed by two applications: 

• One application which has the function of retrieving images from the observation 
cameras (AXIS 214 PTZ), displaying them in real-time and also giving the user the 
possibility to change the orientation and zoom factor of the cameras. 

• The second application is a VNC client. 

The VNC viewer (Java client) is a web teleoperation application which can be executed into 
a web browser. The application connects to the Domino web server which makes a secure 
connection using a TCP/IP tunnel with a server having a private IP address, which cannot 
be accessed from internet but only using the Domino server. The private IP machine has a 
VNC server that exports the display, and also the teleoperation application (see Fig. 7). 
Using the exported display, the user can view and run the application as if the application 
runs on his own computer. The access is made using a username and a password, process 
managed by the Domino server. 

For team training and application development, the system allows accessing related 
documents, presentations and multimedia materials, Web conferencing, instant messaging 
and teamwork support for efficient and security-oriented creation and use of group 
workspaces (students / trainers, researchers). 

The Server and eClients Applications run on IBM PC workstations on which IBM Lotus 
software offerings are customized and integrated with other applications (error-free data 
and command transfer, V+ program editing, real time image display and refresh in multiple 
Client windows, replication functions, Client authentication, etc) in a virtual training and 
research laboratory across geographical boundaries. 
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Lotus-oriented solutions have been considered for transferring messages, status reports, 

data and video camera images, interpreting them and accessing databases created by all 

partners. The final objective of the platform is to develop an E-Learning component 

allowing students to access and download technical documentation, create, test, debug and 

run RV and AVI programs, attend real-time laboratory demonstrations, and check their 

skills in proposed exercises. 

Thus, IBM Lotus software unifies all Application modules, providing the necessary 

management and interconnecting tools for distributed industrial controllers, and the 

collaborative tools with back-end relational databases for team training and research. 

To have access to the system, a user must have a username and a valid password to enter in 

the system. First the user must access the portal site using a Java aware browser (like 

Internet Explorer, Opera, Firefox, with the JRE installed). 

The portal which allows the web access is structured in two zones (Fig. 8): 

• a public zone which contains all documentation, tutorials courses and others, needed by 
users to learn how to exploit the system; this part of the portal can be accessed by anyone; 

• a private zone where the access is based on username and password. The private zone 
gives access to the eClients for teleoperation purposes. 

After entering the correct username and password, the user is allowed in the system and has 
access to the teleoperation application which is a menu driven interface which allows him to 
interact with the system (see Fig. 9). 
The web interface is also composed by two zones: 

• An observation zone (right side in Fig. 9) where the user can monitor the jobs which are 
currently performed by the robots by using three networked cameras (due to the high 
resolution needed, only two cameras can be used at a time); 

• A teleoperation zone where the user can interact with each robot. 
The teleoperation application is composed by three windows (see Fig. 9): 

• A camera control window, where the user can control all the camera parameters (tilt, 
pitch, zoom, focus and iris opening); 

• A command window where the user can select the robot system which he wants to 
control, issue commands from the command line or activate the third window (Vision); 

• A Vision window. 

From the Vision window, vision commands can be issued by selecting the desired actions 
from the menus. The most important available functions are: 

• selecting the physical and virtual cameras, and the virtual image buffers; 

• selecting the display mode and the resolution; 

• image acquisition; 

• executing primary operations (histogram, thresholding, etc.); 
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displaying the vision system status; training models; 
configuring switches and parameters for virtual camera set up. 
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The advantage of the Vision window is that all commands can be issued using menus, but 

also the fact that the image acquired by the camera and sent to the server can now be 

accessed at pixel level. Another major advantage is that the training of part recognition and 

grasping models become a single-step process during which a unique window is used for 

parameters and constraints specification (Tomas Balibrea, et ah, 1997). 

The client application can acquire full or partial images using the VGETPIC V+ operation 

and send them to the server (Adept Technology, 2001). 

Captured image can be processed via the menus (filtering, binarization, convolution, 

morphing, etc.), saved in a common format and sent to a specialized image processing 

application. After processing, the image can be sent back to the client and integrated in the 

image processing module using the VPUTPIC V+ operation, for further use (an application 

using this mechanism was developed, and consists of a part identifying algorithm based on 

skeleton computation and matching. 

In order to execute vision programs the user must set up the vision parameters such that the 

system can "see" the objects with the best image quality. The system has specialized 

functions to automatically establish these parameters or manually if some special settings 

are required. 

After setting the parameters and loading the calibration camera-robot, the user can measure 

objects, apply filters, apply morphological operators, train and recognize objects. 

The measurements include signature analysis; polar and linear offset signatures are 

computed and stored to be further used in applications. The skeleton computation is also 

included in the "Measurements" menu item (Borangiu, et ah, 2005). 

An important feature of the system is the mechanism of training object recognition models 

and multiple grasping positions related to each model in order to provide alternative robot 

poses for collision-free object grasping at runtime. 

Training of object models can be done in two ways: 

• Creating a standard ObjectFinder model which is computed by a high level image 
processing module integrated in the system. The training procedure for this type of 
model is based on a sequence of steps which have been embedded in the application. 

• Selecting a set of object features and storing them in a data structure which will further 
characterize each model (Borangiu, 2004). 

After training and storing models, the user can write applications using these models for 
object search; the interface can be accessed from a great variety of places. 
Therefore great care was taken to ensure optimal use in case of low bandwidth and poor 
quality hardware. High level of security and availability are key components; they were 
ensured by the selection of high quality technical devices and well-planned loading. As the 
portal must be prepared for an increasing number of users, the tool must be highly scalable. 
It needs to integrate contents and services and provide access for document repositories. 
User groups must be able to access personalised contents. 

High-level availability must be also guaranteed. The system must be scalable according to 
load and requirements. The use of NLBS {Network Load Balancing System) provides a solution 
for an even load of the network. The portal user interface needs to be customized; templates 
must be created and uploaded. The authorisations and user groups must be defined. 
The eClient application and Lotus Domino Server implement a security access policy to the 
virtual workspace. The access to the eClient application is granted based on the Domino 
defined ACL's {Access Control Lists), such that the user must specify a user name and a 
password in order to connect to the application. Two classes of privileges were defined: 
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1. A user class where the operator can observe images acquired from observation web 
cameras and images from the Vision system taken by multiple area cameras; he can also 
view the commands issued by the trainer and watch the results of the commands; 

2. A trainer class where the operator is authorized to issue commands for every connected 
robot system, upload, download and modify programs. The trainer can also take pictures 
from an individual camera and use the specific vision tools to process that image. The 
observation cameras can also be moved and positioned in the desired location by the 
trainer. The trainer can give users full or partial permission to program testing. 

The communication between users is realized by help of an integrated console (text mode) 
or using an Instant Messaging and Web Conferencing application (Sametime). 
IBM Lotus Domino server software was used to combine enterprise-class messaging and 
calendar / scheduling capabilities with a robust platform for collaborative applications on a 
wide variety of operating systems. Designing the Lotus Domino Server made available three 
offerings: Domino Messaging Server (messaging only), Domino Utility Server (applications 
only), and Domino Enterprise Server (both messaging and applications) (Brooks, et at., 2004). 
Some of the most important Lotus Domino features were used: 

• Encryption, signing, and authentication using the RSA public-key technology, which 
allows marking a document such that the recipient of this document can decisively find 
out that the document was unmodified during transmission. 

• Access Control Lists (ACLs) determining who can access each database (application) 
and to what extent. 

• Usage of Domino's new features to reduce network utilization. Network compression 
reduced the number of bytes sent during transactions by up to 50 percent. Connections 
across heavily loaded links such as WANs and XPCs saw the most benefit. 

• Availability for the Windows NT and XP platforms, automatic fault recovery after 
shutdown and server restart without administrator intervention after the occurrence of 
an exception. Fault recovery used operating system resources, like message queues. 

Because, the application eClient is accessed over the Internet, security represented a critical 
element of the design. Access to different levels of the application is controlled by xACLs 
(extended ACLs) to allow or disallow access. The existing database Access Control Lists 
(ACLs) and the new ACL file feature ensure that application-private databases remain 
secure. In addition, file protection documents for the Domino Web server used to serve the 
eClient (Java application) provide additional access control for files accessed via HTTP. 

4.3 Resource, resource class, and attribute 

This section provides information about the three terms used in RMC: resource, resource 

class, and attribute, as illustrated in Fig. 10. 

Resource 

The resource is a fundamental concept in the RMC architecture. A resource is an abstraction 

of an instance of a physical or logical entity that provides services to applications or system 

components. A system or cluster is composed of numerous resources of various types. 

Each resource has the following characteristics: 

• An operational interface used by its clients. This interface is the reason for the resource's 
existence. For example, the operational interface of a data file is given by the standard 
open, close, read, and write system calls. 

• A set of persistent data values called persistent attributes, describing the characteristics or 
configuration of the resource (e.g. the file system name, data file name, and so on). 
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• A set of dynamic data values called dynamic attributes, reflecting the current state or 
other measurement values of the resource (e.g. the disk block usage of a file system). 

• A handle that uniquely identifies the resource within the cluster. This value is unique 
across time and space and is called a resource handle. 

• A set of operations that manipulate the state or configuration of the resource. 
Resource class 

A resource class is a collection of resources having similar characteristics. The resource class 
provides descriptive information about the properties and characteristics that are common 
to any resource within the resource class (for example, all the file system resources, files 
such as progl.v2 and prog2.v2 belong to the file system resource class in RMC). 
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Fig. 10. Resource managers, resource classes, resources, and attributes. 

Each resource class has the following characteristics: 

• Descriptive information about the class and its resources. 

• A list of the resource handles for all existing resources within the class. 

• A set of persistent data values called persistent attributes that describe or control the 
operation of the resource class. 

• A set of dynamic data values called dynamic attributes (e.g. every resource class supports 
a dynamic attribute that changes whenever the resource number in the class changes. 

• An access control list (ACL) that defines permissions authorizing users to manipulate or 
query the resource class. 

• A set of operations to modify or query the resource class. 
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Attribute 

A resource class and a resource have several attributes. An attribute has a value and a unique 

name within the resource class or the resource. 

A resource attribute is classified into either a public or a private property. The property is 

used as a hint for the RMC client for whom the attribute is to be presented to general users. 

Private attributes typically contain information which is not relevant to general users; 

therefore private attributes are hidden by default. However, one can display private 

attributes by specifying a flag in the command. Web-based System Manager does not 

support the display or modification of private resource attributes. 

Attributes fall into two categories: persistent and dynamic. Each category has a different role, 

and is used differently by RMC clients: 

• Persistent attributes: persistent attributes are configuration parameters for resources, 
and define the resource's characteristics (examples of persistent attributes: Device 
Name, IP Address, etc). For resource classes, persistent attributes describe or control the 
operations of the class. For the RMC client, the main usage of persistent resources is to 
specify which resources will be accessed. They are used as filters over the set of 
managed resources to select those on which the client wants to act. 

• Dynamic attributes: Dynamic attributes reflect internal states or performance variables 
of resources and resource classes. One generally refers to dynamic attributes to define 
the monitoring condition of the resource to be monitored (for example, if you want to 
monitor a file system resource, you can monitor the disk space usage of the file system). 

4.4 Resource managers 

Each resource manager is the interface between the RMC subsystem and a specific aspect of 
the Adept Windows operating system instance it controls. All resource managers have the 
same architecture and interact with other RMC components. However, due to their specific 
nature they have different usage for the end user. Resource managers fall into four groups: 

1. Logging and debugging (Audit resource manager) The Audit Log resource manager is 
used by other RMC components to log information about their actions, errors, and so 
on. The typical end user of Audit Log resource manager is a system administrator 
which scans these logs to see what is happening in the various RMC components. 
Unless a problem occurs, there is generally no need for the user to look at these logs. 

2. Configuration (Configuration resource manager). The configuration resource manager is 
used by the system administrator to configure the system in a Peer Domain cluster. It is 
not used when RMC is configured in Standalone or Management Domain nodes. 

3. Reacting to events (Event Response resource manager). The Event Response resource 
manager is the only resource manager directly used in normal operation conditions. 
The CLI allows the end user to define which events he wants to monitor in the nodes, 
and how the system should react to these events. 

4. Data monitoring (Host resource manager, File system resource manager, Sensor resource 
manager). This group contains the file system resource manager and the Host resource 
manager. They can be seen by the end user as the containers of the objects and variables 
to monitor. However, it is important to note that these resource managers have no user 
interface. The end user cannot interact directly with them and has to pass through the 
Event Response resource manager to access the resources managed by these resource 
managers. 
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The only exception is the Sensor resource manager. It is the only resource manager that 
provides for expandability. Its user interface therefore contains four commands to let the 
end user define, modify or delete resources to be monitored. Once these resources are 
created, the normal interface to them is through the Event Response resource manager. 

4.4.1 Event response resource manager 

The Event Response resource manager (ERRM) plays the most important role to monitor 
systems using RMC. It uses several terms like action, condition, expression, response which 
will be discussed later. 

The Event Response resource manager provides the system administrator with the ability to 
define a set of conditions to be monitored in the various nodes of the cluster, and to define 
actions to be taken in response to events (Lascu et ah, 2002; Lascu, 2005). These conditions 
are applied to dynamic properties of any resources of any resource manager in the cluster. 
The Event Response resource manager provides a simple automation mechanism for 
implementing event driven actions. Basically, one can do the following actions: 

Define a condition composed of a resource property to be monitored and an expression 

that is evaluated periodically. 

Define a response that is composed of zero or several actions that consist of a command 

to be run and controls, such as to when and how the command is to be run. 

Associate one or more responses with a condition and activate the association. 
In addition: 

Different users can employ the same conditions and responses, mix and match them to 

build different monitoring scenarios. 

Many predefined conditions and responses are shipped with the system and can be 

used as templates for the user who wants to define own monitoring scenarios. 

Any events that occur and any responses that are run are logged in the audit log. One 

can look at the audit log to check that the defined conditions were triggered, how the 

system reacted, and what was the result of the actions. 
The Event Response resource manager provides one with both a CLI and a GUI (Web-based 
System Manager). It can then be decided whether you want to use the Web-based System 
Manager or the CLI. In a stand-alone environment and for setting up limited monitoring of 
the resources, the Web-based System Manager is the easiest tool to use, unless the user is 
already familiar with the CLI commands and arguments. However, for managing a cluster 
or developing sophisticated monitoring scenarios, the recommended tool is the CLI where 
the user can write his own CLI commands. 
Expression 

The expression is the means of making computations with data stored in attributes and 
audit log record fields. An expression is composed of variables, constants and operators, 
and returns a Boolean value: true or false. 

The same syntax is used regardless of the following three environments; however, the 
expressions handle different types of variable: 

• ERRM conditions: trigger events based on dynamic attributes of resources. 

• GUI or CLI: filter resources based on variables representing persistent attributes of the 
resources. 

• Audit log expressions: select audit records with variables being the name of a record 
field. 
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Condition 

A condition applies to a resource class. It may contain a selection string, which is used to 
define which resource in the resource class should be monitored. It also contains a Boolean 
expression that must be satisfied by one (and only one) dynamic attribute of the resource to 
fire the condition. A condition has a name, so the user can refer to it once defined. A 
condition can also contain an optional rearm expression. 

Fig. 11 shows the relationship between the monitored resource(s), conditions, events, and 
rearm events. 
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Fig. 11. Condition, expression, and events. 

For example, to see if the file system uses more than 90 percent of its available disk space, a 
predefined condition called "disk space used" is provided with RMC. This condition uses 
the file system resource class and selects the local disk file system from all the file systems of 
the monitored nodes. It contains the expression "PercentTotUsed > 90" which applies to the 
PercentTotUsed attribute of the file system resource. 
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A condition may also contain a node group list that the event is monitored on. The default 

value for the node group list is NULL for all the nodes within the management scope. The 

default value for the management scope is local scope for the local node. 

A condition can specify multiple monitored resources selected from a resource class by the 

selection string. A condition cannot select resources selected from multiple resource classes. 

Multiple conditions can specify the same resource(s); however, they must have different 

condition names. If multiple conditions that specify the same resource (s) have the same 

expression or the rearm expression, then they define the same event or the rearm event. 

However, you may want to specify different actions for them. 

There are no objects defined as event and rearm event. If an event expression is evaluated as 

true, the result is called as an event. If a rearm expression is evaluated as true, the result is 

called as a rearm event. 

Action 

RMC comes with three predefined actions: logevent, notify event, and wallevent: 

• The logevent action appends a formatted string to a user defined log file that can be read 
later on using the alog command. 

• The notifyevent action sends the content of the event to the user. 

• The wallevent event broadcasts a message to all users. 

One can specify when an action should be executed from as follows: 

• Only when the event defined by the associated condition occurs. 

• Only when the rearm event defined by the associated condition occurs. 

• Either the event or the rearm event defined by the associated condition occurs. 
Finally, the user can also provide customized scripts as actions and the existing scripts can 
be used as examples or starting points for more complex operations. User defined scripts are 
executed in an environment where variables are set according to the event that would 
trigger the action. For example, the ERRM_RSRC_NAME variable would contain the name 
of the resource whose dynamic attribute change caused an event to fire. 

Response 

A response is a set of zero or more actions that will be performed by the system when an 
expression of a condition evaluates to true. A response contains the following information 
on how the actions are executed: 

• Arguments for the actions. 

• Time ranges (periods) to describe when the action can be run. 

• Definitions of environment variables to set before executing the action. 

• The expected return code of the action and whether the results of the action should be 
directed, for example, to the audit log. 

A response can be defined with multiple actions. For example, the critical notification 
response calls the following three predefined actions: 

• Logging an entry in the criticalEvents.log. 

• Sending a message to the user. 

• Warning all logged users with wallevent. 
Association between a condition and a response 

Conditions and responses can exist without being used and with nothing related to each 
other. Actions are part of responses and only defined to responses. Although it is possible 
that multiple responses have an action using the same name, these actions do not refer to the 
same object. One can associate a response with multiple actions. 

To start observing the monitored resource, a condition must be associated with at least one 
response. A condition can be associated with multiple responses. However, each pair of a 
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condition and a response is considered a separate association. Once associated, the condition 
and response should be activated (started). 

Fig. 12 illustrates the relationship between the conditions, the responses, and the actions. In 
this scheme, there are three associations (A, B, and C). 
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Fig. 12. Conditions, responses and actions. 

An association has no name. The labels A, B, and C are for reference purposes. To refer to 
the specific association, one must specify the condition name and the response name that make 
the association. For example, one must specify the condition 1 and the response 1 to refer to 
the association A. Also, it must be clear that the same action name (in this example, action a) 
can be used in multiple responses, but these actions are different objects. 
ERRM evaluates the defined conditions which are logical expressions based on the status of 
resources attributes; if the conditions are true a response is executed. 



4.4.2 Audit log resource manager 

The Audit Log resource manager provides the other RMC subsystems with a logging 
facility. It can be seen as a means to trace various actions taken by the cluster system. 
Typically, these subsystems will add entries in the audit log corresponding to their normal 
behaviour or to error cases. Examples of information provided in the audit log are: 

Instances of starting and stopping monitoring 

Events 

Actions taken in response to events 

Results of these actions 

Subsystem errors and monitoring errors 
Only subsystems can add records to the audit log; there is no user interface that allows the 
end user to add entries in the audit log. 
Only two actions are offered by the Audit Log resource manager: 

• Lsaudrec: List records in the audit log. 

• Rmaudrec: Delete the specified records in the audit log. 



4.4.3 Configuration resource manager 

The configuration resource manager can create and manage a peer domain cluster. Thus, the 
configuration resource manager need not be used to use RMC on a stand-alone server or in 
a management domain. 
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4.4.4 File system resource manager 

The File system resource manager manages only one resource class: the local file systems on 
the cluster node. It can be used to list the file systems, get their status, and retrieve values 
like the percentage of used space or inodes. 

4.4.5 Host resource manager 

The host resource manager provides the facility to monitor hardware devices and programs 
running on the cluster nodes. The monitored hardware includes disks, serial lines and 
network interface adapters. The program monitoring feature allows you to monitor when 
processes executing any arbitrary program are created or destroyed. 

4.4.6 Sensor resource manager 

The sensor resource manager allows the user to extend the functionality of RMC for 

monitoring parts of the Adept Windows robot controller operating system for which RMC 

does not provide a predefined resource class or attribute. 

The sensor resource manager provides four commands to create, rename, or remove new 

resources called sensors. A sensor is a command that will be periodically executed, returning 

a value that can be monitored by defining a condition in the ERRM. By default, the period of 

execution is 60 seconds, but this period can be changed by the user to another value when 

creating the sensor. 

If more than one returned value is needed, it is possible to define sensors returning multiple 

"property = value" pairs. This feature can be used, for example, to implement behaviours, 

such as notify user of the value of properties 2 and 3 when property 1 exceeds a threshold. 

5. Solution implementing for networked robots 

In order to implement the solution on a network of robot controllers, first a shared storage is 

needed, which must be reached by any controller from the cluster. 

The file system from the storage is limited to NFS (network file system) by the operating 

system of the robot controllers (Adept Windows). Five Adept robot manipulators were 

considered, each one having its own multitasking controller. 

For the proposed architecture, there is no option to use a directly connected shared storage, 

because Adept robot controllers do not support a Fiber Channel Host Bus Adapter (HBA). 

Also the storage must be high available, because it is a single point of failure for the 

Fabrication Cluster (FC), see Fig. 13. 

Due to these constraints, the solution was to use a High Availability cluster to provide the 

shared storage option (NFS Cluster), and another cluster composed by Adept Controllers 

which will use the NFS service provided by the NFS Cluster (Fig. 13). 

The NFS cluster is composed by two identical IBM xSeries 345 servers (2 processors at 2.4 

GHz, 1GB RAM, and 75GB Disk space, two RSA 232 lines, two Network adapters, and two 

Fiber Channel HBA), and a DS4100 storage. The storage contains a Quorum volume used by 

the NFS cluster for communication between nodes, and a NFS volume which is exported by 

the NFS service which runs in the NFS cluster. The servers have each interface (network, 

serial, and HBA) duplicated to assure redundancy (Anton et ah, 2006; Borangiu et ah, 2006). 

In order to detect the malfunctions of the NFS cluster, the servers communicate using 

heartbeat packets to ensure that the communication is established. 
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MANUFACTURING STRUCTURE (Cell, Line, ...) 
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Fig. 13. Implementing the high availability solution for the networked robotic system. 

There are three communication routes: the first route is the Ethernet network, the second is 
the quorum volume and the last communication route is the serial line. If the NFS cluster 
detects a malfunction of one of the nodes and if this node was the node which served the 
NFS service the cluster is reconfiguring as follows: 

1. The server which is still running writes in the quorum volume which is taking the 
functions of the NFS server, then 

2. Executes the resource takeover operation (the resource being the NFS volume) and 
mounts the NFS volume, then 

3. Takes the IP of the other server (IP takeover) and 

4. Starts the NFS service. 



An Implementation of High Availability in Networked Robotic Systems 65 

In this mode the Fabrication Cluster is not aware about the problems from the NFS cluster, 

because the NFS file system is further available. 

The Fabrication Cluster can be composed by at least two robot controllers (nodes) - group 

leader and group leader backup. The nodes have resources like: robot manipulators (with 

attributes like: collision detection, current robot position, etc.), serial lines, Ethernet 

adapter, variables, programs, and NFS file system. The NFS file system is used to store 

programs, log files and status files. The programs are stored on NFS to make them available 

to all controllers, the log files are used to discover the causes of failure and the status files 

are used to know the last state of a controller. 

In the event of a node failure, the production flow is interrupted. In this case, if there is a 

connection between the affected node and the group leader, the leader will be informed (if 

not, the heartbeat detects the failure) and the GL takes the necessary actions to remove the 

node from the cluster. The GL also reconfigures the cluster so the manufacturing process 

will continue. For example if one node cluster fails in a three-node cluster, the operations 

this node was doing will be reassigned to one of the remaining nodes. 

The communication paths in the multiple-robot system are the Ethernet network and the serial 

network. The serial network is the last communication resort due to the low speed and also 

to the fact that it uses a set of Adept controllers to reach the destination (because it is a ring 

type connection). In this case the ring network will be down if more than one node fails. 

Rebuilding the communication route in the network represents the last step required to 

restart the normal behave of the FC control system. 

In case of malfunction of the communication network the following important cases can appear: 

1. If the connection between the Switch and the Supervisor PC (the PC where the Server 
Application runs) is down the remote control will be lost, but the FMC will reconfigure 
as follows: the controller will use the Ethernet network for communication, and the 
controller with the first IP from the class will take the functions of the Supervisor PC. If 
the connexion is re-established the Supervisor PC makes a query, finds the replacing 
controller, transfers the databases and restarts the normal behaviour. 

2. If the switch is not functioning, all the Ethernet connexions are lost, but the controllers 
will use the serial "network". The behaviour is like in the first case only that the web 
users can view the status from the Supervisor PC, including the images acquired by the 
observation cameras. 

3. If a controller loses the Ethernet connexion, it will use one of the two serial lines to 
reach the Supervisor PC depending on the CPU time of the neighbours. 

6. Conclusion 

Fault-tolerance is provided to the cell communication system (Fig. 14), and therefore 
redundancy at both Station Controller level (a break down of a Robot Controller is 
detectable, the production tasks can be rescheduled to the remaining valid units for graceful 
degraded behaviour) and Station Computer level (replication of data bases for the IBM PC- 
type device terminals, reassignment of computers in case of break downs). 
The fault tolerance solution presented in this paper is worth to be considered in 
environments where the production structure has the possibility to reconfigure, and where 
the manufacturing must assure a continuous production flow at batch level (job shop flow). 
There are also some drawbacks in this solution, like the need of an additional NFS cluster. 
The spatial layout and configuring of robots must be done such that one robot will be able to 
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take the functions of another robot in case of failure. If this involves common workspaces, 
programming must be made with much care using robot synchronizations and monitoring 
continuously the current position of the manipulator. 
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Fig. 14. Fault-tolerant communication architecture. 

The advantage of the proposed solution is that the structure provides a continuous 

production flow with an insignificant downtime (during reconfiguration). 

The solution was tested on a five-robot assembly cell located in the Robotics and IA 

Laboratory of the University Politehnica of Bucharest. The cell also includes two CNC 

milling machines and one Automatic Storage and Retrieval System, for raw material feeding 

and finite products storage. 

During the tests, the robot network has detected a number of errors (end-effector collision 

with parts, communication errors, power failure, etc.) The robotic network has evaluated the 

particular situation, and the network was reconfigured and the abandoned applications 

were restarted in a time between 0.2 and 3 seconds. The network failure was also simulated 

during tests (Fig. 15). 

One robot (R2) was disconnected from the Ethernet network, the heartbeat packet sent by 

the robot to the other cluster members has detected the malfunction and the robot has 

switched the communication using the serial line; this was done in 0.3 seconds after the 

Ethernet cable was removed. The communication between the affected robot and its 

neighbours was done using the serial lines, and the communication with other robots was 

done by routing the communication using the Ethernet line of the neighbours (Rl and R3). 

In this way the communication latency was reduced. 

After the communication was re-established, the serial lines of the robot were disconnected. 

The robot has detected the communication failure, stopped the manipulation program and 

retracted the manipulator in a home position in the exterior of the working area in 0.8s. 

The neighbours have sent the heartbeat packets using the serial lines and detected that they 

do not have any connection with the robot and announced the group leader (GL) which has 
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removed the robot from the cluster and reconfigured the cell (see Fig. 15). The neighbour Rl 
having the same working area as the affected robot R2 has loaded the values of the variables 
and the production program from the shared storage, and started production, continuing 
from the point where R2 has been stopped. The cell reconfiguration from the point where 
the serial lines were disconnected has taken 2.2 seconds. 
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Fig. 15. Algorithm for network failure detection and reconfiguration. 

Another communication test consisted in disconnecting both the serial lines and the 
Ethernet line at the same time, in this case the cluster tested the communication sequentially 
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and the cluster reconfiguration took 2.5 seconds. When configuring the cluster in order to 

test in parallel the communication lines, the reconfiguration took 2.3 seconds but the 

controllers processed a higher communication load. 

The most unfavourable situation occurred when a robot manipulator was down; in this case 

the down time was greater because the application which was executed on that controller 

had to be transferred, reconfigured and restarted on another controller. Also if the controller 

still runs properly it will become group leader to facilitate the job of the previous GL (that 

has also a manipulation task to do). 

The proposed solution is not entirely fault tolerant; however, in some situations it could be 

considered as a fault tolerant system due to the fact that even if a robot controller failed, the 

production continued in normal conditions. 

The project is in the final stage of development and will be finished at the end of 2008. The 

research project will provide a portal solution for linking the existing fault tolerant pilot 

platform with multiple V+ industrial robot-vision controllers from Adept Technology 

located in different labs. 
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1. Introduction 

Diagnosability property and fault detection schemes have been widely addressed on 
centralized approaches using the global model of the Discrete Event System (DES). Roughly 
speaking, diagnosability is the property of determining if using the system model is possible 
to detect and locate the faulty states in a finite number of steps. In the works (Sampath, et 
al., 1995) and (Sampath, et al., 1996), a method for modeling a DES using finite automata is 
proposed; based on this model, a diagnoser is derived. The cycles in the diagnoser are used 
to determine when the DES is diagnosable. 

Recently, fault diagnosis of DES has been addressed through a distributed approach 
allowing breaking down the complexity when dealing with large and complex systems 
(Benveniste, et al, 2003; O. Contant, et al, 2004; Debouk, et al, 2000; Gene & Lafortune, 
2003; Jiroveanu & Boel, 2003; Pencole, 2004; Aramburo-Lizarraga, et al., 2005). 
In (Debouk, et al., 2000) it is proposed a decentralized and modular approach to perform 
failure diagnosis based on Sampath's results (Sampath, et al., 1995). In (Contant, et al., 2004) 
and (Pencole, 2004) the authors presented incremental algorithms to perform diagnosability 
analysis based on (Sampath, et al., 1995) in a distributed way; they consider systems whose 
components evolve by the occurrence of events; the parallel composition leads to a complete 
system model intractable. In (Gene & Lafortune, 2003) it is proposed a method that handles 
the reachability graph of the PN model in order to perform the analysis similarly to 
(Sampath, et al., 1995); based on design considerations the model is partitioned into two 
labelled PN and it is proven that the distributed diagnosis is equivalent to the centralized 
diagnosis; later, (Gene & Lafortune, 2005) extend the results to systems modeled by several 
labelled PN that share places, and present an algorithm to determine distributed diagnosis. 
In (Qiu & Kumar, 2005) it is studied the codiagnosability property, this property guarantees 
that any faults occurred in the system must be detected by at least one local diagnoser in a 
finite number of steps using the local information, besides, a notion of safe-codiagnosability 
is mentioned to capture the fact that the system has a safe specification while the system 
performance is tolerable. (Aramburo-Lizarraga, et al., 2005) proposes a methodology for 
designing reduced diagnosers and presents an algorithm to split a global model into a set of 
communicating sub-models for building distributed diagnosers. The diagnosers handle a 
system sub-model and every diagnoser has a set of communication events for detecting and 
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locating the faults of the corresponding sub-model. (Aramburo-Lizarraga, et al., 2007) 
shows how to design low interaction distributed diagnosers reducing the communication 
among them and proposes a redundant distributed diagnoser scheme composed by a set of 
independent modules handling two kinds of redundancy (duplication or TMR). 
This work considers the system modeled as an interpreted PN (IPN) allowing describing the 
system with partially observable states and events; the model includes the possible faults it 
may occur. In order to build such a model, this work presents a bottom-up modeling 
methodology in which the behavior of the system elements is decomposed into state 
variables; a range for each state variable must be settled. These ranges represent the possible 
values of state variables. Afterwards, these rages are coded into IPN (modules), where each 
value is represented by a different place. Then two composition operators for joining 
modules are used; the first one is named synchronic composition, which merges transitions 
according to certain rules. It is similar to the synchronic product presented in (Giua & 
DiCesare, 1994); the second one is named permissive composition, which uses selfloops for 
enabling transitions among modules; this allows constraining the model behavior into the 
actual system behavior. This new operator avoids the use of tuning phases of other 
modeling methods used in supervisory control (Giua & DiCesare, 1994). Based on the 
derived IPN model with the proposed methodology, the diagnosability property for IPN 
models is introduced. Roughly speaking, this property says that an IPN is diagnosable if it is 
possible to know both, when a faulty place is marked and which faulty place is marked. 
This property is closely related to the observability property (Aguirre-Salas, et al., 2002) and 
(Ramirez-Trevino, et al., 2003) and polynomial algorithms to test when an IPN is 
diagnosable are derived, avoiding the reachability analysis of other approaches. Also, a 
distributed diagnoser is presented; every distributed diagnoser uses the local information or 
communication among diagnosers for detecting and locating a system fault. The 
diagnosability property is preserved in the distributed architecture. Redundancy techniques 
could be applied to the distributed diagnosers to detect and locate a malfunction in the 
distributed diagnosers set. 

The chapter is organized as follows: section 2 provides basic definitions of PN, IPN and the 
modeling methodology are presented. In section 3 the property of input-output 
diagnosability is defined and characterized, a diagnoser scheme devoted to detect and 
isolate failure states is also presented. Section 4 presents a procedure to build a reduced IPN 
model. Section 5 describes a method for model decomposition allowing interaction 
distributed diagnosers, also, it is presented a redundant scheme for reliable diagnosis 
applying redundancy to the distributed diagnosers. Finally, conclusions are given. 

2. Basic notations and system modeling 

2.1 Petri net basics 

We consider systems modeled by Petri Nets and Interpreted Petri Nets. A Petri Net 
structure is a graph G = (P, T, I, O) where: P = {pi, pi, ..., p n } and T = {ti, h,... ,t m } are finite 
sets of nodes called respectively places and transitions, I (O): P x T — > 1 + is a function 
representing the weighted arcs going from places to transitions (transitions to places), where 
I + is the set of nonnegative integers. 

The symbol *tj (tj ) denotes the set of all places p\ such that !(/?;, £ ; )^0 (O(/?i,£ ; )^0). Analogously, 
'pi (pi*) denotes the set of all transitions tj such that O(/?;,£ ; )^0 (!(/?;, £ ; )^0) and the incidence 
matrix of G is C=[cy], where Qj = 0(pi,tj) - I(pi,tj). 
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A marking function M: P— >^ + represents the number of tokens (depicted as dots) residing 
inside each place. The marking of a PN is usually expressed as an n-entry vector. 
A Petri Net system or Petri Net (PN) is the pair N=(G,Mo), where G is a PN structure and Mo 
is an initial token distribution. R(G,Mo) is the set of all possible reachable markings from Mo 
firing only enabled transitions. In a PN system, a transition tj is enabled at marking Mk if Vp,- 
e P, Mkipi) ^ I(pi,tj); an enabled transition tj can be fired reaching a new marking Mu+i which 
can be computed as Mk+i = Mk+ Cvk, where Vk(i)=0, vt], Vk(j)=l. 

Interpreted Petri Nets (IPN) (Ramirez-Trevino, et al., 2003) is an extension to PN that allow 
to associate input and output signals to PN models. An IPN (Q, Mo) is an Interpreted Petri 
Net structure where Q = (G, Z, X, cp) with an initial marking Mo, G is a PN structure, 2 = {ai, 
ci2, ... At) is the input alphabet of the net, where fl; is an input symbol, X: T— >Z u{e} is a 
transition labelling function with the following constraint: \/tptk e T, ; ± k, if Vp,- I(pi,tj) = 
I(pi,h) ^ and both \(tj) ^ e, X(tk) ± e, then X(tj) + X(tk)', e represents an internal system event, 
and cp\ R(Q,Mo)^>( % + )^ is an output function that associates to each marking an output 
vector, where a is the number of outputs. In this work cp is a q*n matrix. If the output 
symbol i is present (turned on) every time that M(j)j)>l, then <p (i,j)=l, otherwise ^(z,;)=0. 
A transition tj e T of an IPN is enabled at marking Mk if Vp,- e P, Mkipi) ^ I(pi,tj). If tj is 
enabled at marking Mk, and X(tj) is present, then tj can be fired reaching Mk+i, i.e., 
M \ — - — >M k+l ; Mk+i can be computed using the state equation: 

M k +i = M k + Cv k , . 

y k = (p(Mk) ( } 

where C and Vk are defined as in PN and yk e (^ + ) < i is the k-th output vector of the IPN. 

The sequence a — tt ....t k ... is a firing transition sequence of an IPN (Q,Mo) if 

M — ^^Mj — tj —>..M x — *•*—>... According to functions X and <p, transitions and places of 
an IPN (Q,Mo) are classified. If X(U) ± s the transition U is said to be manipulated. Otherwise 
it is non-manipulated. A place pi e P is said to be measurable if the i-th column vector of cp 
is not null, i.e. cp (»,i) ^ 0. Otherwise it is non-measurable. The following concepts are useful 
in the study of the diagnosability property. 

The set £(QMo) = {cr = t.t...J k ... a M — tj —>M l — tj —>..M x — tj —>... } of all firing 
transition sequences is called the firing language of (Q,Mo). 

A sequence of input-output symbols of (Q,Mo) is a sequence go = (ao,yo)(ai,yi)...(a n ,yr), where 
fly e 2 u{s}. The symbol m+i is the current IPN input when the output changes from y\ to yi+i. 
It is assumed that ao = e and yo = <p(Mo). 

The firing transition sequence o e £(Q,Mo) whose firing actually generates go is denoted by 
o ro . The set of all possible firing transition sequences that could generate the word go is 
defined as Q(go) = {o | o e £(Q,Mo) a the firing of o produces go}. 

The set A(Q,Mo) = {go | go is a sequence of input-output symbols} denotes the set of all 
sequences of input-output symbols of (Q,Mo) and the set of all input-output sequences of 
length greater or equal than k will be denoted by A k (Q,Mo), i.e. A k (Q,Mo) = {go e A(Q,Mo) I 
| go | > k) where k e M. 
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The set A B (Q,M ) = {(D e A(Q,M ) | ogQ(cd) such that M — Z—>M. and My enables no 

transition, or when M — tj —> then c(»,r)= } denotes all input-output sequences 

leading to an ending marking in the IPN (markings enabling no transition or only self-loop 

transitions). 

An IPN (Q, Mo) described by the state equation (1) is event-detectable iff the firing of any 

pair of transition t;, t ; of (Q, Mo) can be distinguished from each other by the observation of 

the sequences of input-output symbols. 

The following lemma (Rivera-Rangel, et al., 2005) gives a polynomial characterisation of 

event-detectable IPN. 

Lemma 1: A live IPN given by (Q,Mo) is event-detectable if and only if: 

1. Mti, tj e T such that X(U) = X(tj) or X(U) = fit holds that (pC(;t t ) * (fC(;tj) and 

2. \/t k e T it holds that <jpC(;t k ) * 0. 

2.2 System modeling 

We work with the modeling methodology proposed in (Ramirez-Trevino, et.al.,2007). The 
methodology follows a modular bottom-up strategy. After identifying the system 
components, a set of state variables is assigned to every component, each state variable 
behavior is modeled by a PN model, herein named module. Then the set of modules are 
integrated into a single model according to the appropriate relationships achieved through 
two module composition operations. This methodology builds binary IPN modules to 
represent the behavior of each component of the identified DES and the relationships 
between them. The model captures the normal and faulty behavior of the individual 
components of the system. Below it is described the detailed steps of the methodology. 

2.2.1 Modeling methodology 

1. System components.- The system components must be identified and named. 
Afterwards, a finite set System_Components= {sci, SC2, . . ., sc n ] of these names must be 
created. A system component could be a valve, a motor, a system resource, etc. 

2. State variables.- For each system component, the different variables needed to represent 
its behavior must be chosen. In other words, the finite set State_Variablesi= {svh, svh, . . ., 
sv^m} associated to the system component SQe System_Components must be built. These 
variables could represent the position (for instance a valve position), velocity, voltage, 
etc. of each system component, or could represent a task descriptor (for instance a 
machine state). There exists at least one state variable for each system component. 

3. Set of values.- For each state variable sv j j e State_Variablesi, the set 
Value v , = \yal l { ,vall ,...,val u p j of possible values of sv j j must be stated. Necessary faulty 

values should also be considered in this set. For instance, the variable "valve_position" 
may take four values: "Open", "Closed", "ErrorOnOpen" and "ErrorOnClosed", or the 
variable "task_machinel" may take three values: "loading", "processing", and 
"unloading". 

4. Codification.- The values in each set Value . must be represented in terms of PN 

markings. This can be easily achieved if binary places are used. Thus, for each sv j j e 
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State Variables i a set P , = \p* p l I ,..., p u \ of places such that \Value , = \P , must be 

— — svj v* 1 7 -* ^ 7 7 r n ) -T I svj\ \ svj\ 

created. The marking of these places is binary and mutually exclusive. Then, M(pv z )=l 
means that the variable sv j j takes the value vol] . Because of the existence of faulty 
values, the set of places can be partitioned into the subsets P^ and P* , representing 

the faulty and normal values respectively. 

5. Event modeling.- For each pair of values val lJ m , val lJ n such that the state variable sv j j 
could change from value val l] m to a value val l] n , a transition f mn must be created. Then, 
one arc going from place p lJ m to transition f mn and one arc going from transition f mn to 
place p lJ n must be added. 

6. Initial marking.- The initial marking is defined as: M Q \p lJ m )= 1 if the initial value of the 

variable sv j j is val l] m and M \p lJ m ) = otherwise. 

7. Output.- The output of this algorithm is a set of isolated PN modules, each one 
modeling the behavior of a state variable sv j j . 

8. Perform synchronous composition and permissive among modules IPN to obtain the 
global IPN model system (see Alcaraz-Mejia et al. 2003) and (Ramirez-Trevino et al. 
2007). 

In the IPN system model the sets of nodes are partitioned into faulty nodes (P F , places 
coding faulty states, and T F transitions leading to faulty states) and normal functioning 
nodes (PN and 7™); so P = P F u P^ and T = T^uT^. p" denotes a place in PK Since P^ c 

P then p* also belongs to (Q, Mo). The set of risky places of (Q, Mo) is P R = *T F . The post- 
risk transition set of (Q, M ) is T K = P R0 n T N . (q n ,M") denotes the embedded normal 
behavior of (Q, Mo), i.e., (jQ^ ,M ^ J is the subnet induced by normal nodes. 
Example 1. Consider the producer-consumer scheme depicted in figure 1. 



Producer 




Buffer of 2 -slots 



Fig. 1. Producer-Consumer with buffer of 2-slots scheme 

The model consists of a producer unit (PU), a consumer unit (CU) and a buffer of 2-slots. 
The behavior of this system is the following. The producer unit PU creates and delivers 
products into the free buffer positions. The consumer unit CU retrieves products from the 
buffer when there is a product stored into a buffer slot. The producer unit PU could reach a 
faulty state from its producing state. Similarly, the consuming unit could reach a faulty state 
from its consuming state. We model each system component isolately (see figure 2), then the 
synchronous composition and permissive operations are applied to obtain the global system 
model of the producer-consumer scheme. In figure 3 is depicted the normal system behavior 
model; figure 4 includes the faulty behavior, in this case when PU stops its production and 



74 



Advances in Robotics, Automation and Control 



CU stops its consumption. The places pi, p2, p3 represent the normal PU behavior and pn 
represents the faulty behavior. Places p4, ps, P6 represent the normal CU behavior and pi2 
represents the faulty behavior. The places pz, ps, P9 and pio represent the 2- slots of the 
buffer. Function X is defined as A(ti)=a, A(t8)=b; and A(ti)=e for others transitions. 
Measurable places are p 3 , p6, ps, pio, P R = {p3,p6}, T R = {ti,t 8 }, T F = {t 9 ,tio} and P F = {pn,pi2}. 



3. Centralized diagnosability 

The characterisation of input-output diagnosable IPN is based on the partition of R(Q,Mo) 
into normal and faulty markings where all the faulty markings must be distinguishable from 
other reachable markings. 
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Fig. 2. IPN modules of the identified components 
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Fig. 3. Normal behavior of the IPN of the producer-consumer scheme 

p 9 




Fig. 4. Normal and Faulty behavior of the IPN of the producer-consumer scheme 
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Definition 1: An IPN given by (Q,Mo) is said to be input - output diagnosable in k < go steps if 
any marking M/ e F is distinguishable from any other Mk e R(Q,Mo) using any word go e 
A k (Q,M/) u A B (Q,M/), where F = {M | 3p fc e P F such that M(^)>0, M e R(Q,M )}. 
The following result provides sufficient structural conditions for determining the input- 
output diagnosability of an IPN model. 

Theorem 1: Let (Q,Mo) be a binary IPN, such that \Q N ,M" ) is live, strongly connected 
and event detectable. Let {X h ...,X T } be the set of all T-semiflows of (QMo). If V/?" e P N , 
(/?^ ) n T f t^ the following conditions hold: 

1. Vr, 3j X r (j)>l, where f ; - e (p* )* - T F , 

2. Vfe e (/^ )* - T F , \h)={ p^ } and X(t k ) * e. 
then the IPN (Q,Mo) is input-output diagnosable. 

Proof: Assume that (Q,Mo) meets all conditions of the theorem. Since [Q N 9 M* ) is live, then 

it is live by places (see Desel & Esparza, 1995), i.e. Vp e P there exists a marking Mk such 

that Mfc(pi)>0, previous observation is also valid for any pf e P N , (p* ) n T F ¥" 0. Thus 

there exists o Q such that M — ^-^M^and M k (p*) = l. The input-output symbol word 

generating o ro will be denoted by go. Choose the longest transition firing sequence o ro not 
including transitions in T F (could be the empty one). Such sequence must be marking a place 

p* e P N , [p* ) c\F F + ( Z) since in the next step a faulty transition should be fired. 

Since the initial marking is known, [Q N ,M" Jis event detectable, and o Q contains no faulty 

transitions, then the marking Mk such that M — ^-» M \ , can be computed using the IPN 

state equation. 

Since p* has a token at Mk, then a transition f e (p* J n T F could be fired reaching a 

faulty marking M ; . Thus, we will prove that it is possible to detect that M, is reached. 

Assume that such transition f at Mk. Thus M — °-^-^M k — ^-^M and M ; adds a token 

into a place /?*" e (p* J* n P F and removes a token from the place p* . Since the use of the 
modeling methodology assures that A[t^ )— s and (p[M k ) = <p\M ) , then the firing of f 

cannot be detected using the current sequence of input-output symbols, i.e. it is not possible 
to determine when the faulty marking M ; was reached. In other words, the input-output 
symbol word go generates both o ro and o Q f . In order to detect when M ; is reached, we 
proceed as follows. 
Since it holds that Vr, 3j X r (j)>l, tj e T - T F , then no T-semiflow can be fired without firing a 

transition in [p* ) - T F . Moreover, since the number of transitions in (Q,Mo) is finite, then 

the length of a sequence attempting to fire a transition (p* ) should be finite. Then a 

transition tk e [p* J n T N must be attempted to fire after the firing of a finite sequence. 
Since the current marking in the net is the faulty marking M/, then tk cannot be fired. Since 
the condition 2 indicates that A(t k ) ^ e , then we can detect when the symbol X\t k jis given to 
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the system. Moreover, since "(fe)= \P* ) > if fe cannot be fired, then certainly that 
M.(p?)= and that M.(p F ) = 1 , where p F e (/?f)" n P f . Thus the faulty marking My is 
detected, i.e. the IPN is input-output diagnosable. □ 

3.1 Diagnosability test 

Determining when an IPN is input-output diagnosable is reduced to the following tests. 

1. Binarity of (Q,Mo). It is fulfilled because of the modeling methodology. 

2. Liveness of modules can be tested efficiently; however the property is not preserved 
during arbitrary module composition operators. Thus some constraints in the 
application of synchronous and permissive compositions are introduced to guarantee 
liveness during composition. Such constraints may follow the rules introduced in (Koh 
& DiCesare, 1991) for module composition. 

3. Event detectability and strongly connectedness are determined in polynomial time as 

well as detecting places p* e P N such that (p* ) c^T F i" 0. Then condition 2 of previous 
theorem is efficiently tested. 

4. Finally, condition 1 can be verified in polynomial time. In this case we need to check 

that there exists no T-semiflow that does not include transitions in \p* ) , p" e P N such 

that [p* ) n T F + 0. Thus we need to check that the following linear programming 
problems have no solutions. 
V/?" e P^such that pf e P N , (p? )' nT^0 

ax 

s.t. 
CX=0 

x(j)=0,V^(^)*-Tf 

There exist different strategies for constructing the diagnoser-model, we presented two 
different ways: a centralized and a distributed diagnoser. A centralized diagnoser model is 
composed by a copy of the normal system behavior of (Q,Mo). Also a reduced diagnoser 
model can be built. Also, this work handles a distributed diagnoser which is very useful for 
large and complex system besides incorporate reliability to the system diagnosis process 
through redundancy techniques applied to the distributed diagnoser model. The different 
diagnoser models are defined in the following sections. 

3.2 Centralized diagnoser design 

Diagnosability theorem given above provides the basis for designing an on-line diagnoser 
(see figure 5). Such diagnoser can detect a fault and locate faulty markings reached by an 
IPN. The proposed scheme for diagnosis (Ramirez-Trevino, et al., 2004) handles a copy of 
the normal behavior model which must evolve similarly to the system; the outputs of both 
the system and the model are compared and, when there is a difference, a procedure is 
started to compute the faulty marking. 

Example 2. The IPN system model depicted in figure 3 represents the normal behavior 
model for the producer-consumer diagnoser. The initial marking Mo represents that all the 
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buffers are empty, the PU is waiting to deliver and the CU state is idle. Since this IPN is 
input-output diagnosable by theorem 1, then we can detect and locate the fault with the 
diagnoser. The system and its on-line diagnoser are depicted in figure 6, notice the 
diagnoser is a copy of the system. Now assume that the events represented by the sequence 
tih are executed into the system, then the sequence hh is fired in the diagnoser model. Thus 
both, the system and the diagnoser-model have the same output "producing" and 
"consuming". If the fault transition tu is fired, then pu is marked in the system and no 
system output change is detected. When the symbol of h, X(ti) is given as input to the 
system, then the diagnoser-model evolves and its new output is "consuming", however the 
output of the system continues in "producing" and "consuming". Then the difference is the 
signal "producing"; thus the algorithm determines that pn is marked, thus the fault is 
isolated. 

System 

<P(M k ) 

~ Output of (Q,MJ 



Input 



(Q,Mo) 




Diagnoser-model 



(Qi f MJ>) 



Diagnoser 

Fig. 5. On-line diagnose scheme 
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Fig. 6. On-line diagnoser model 
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4. Reduced diagnoser 

4.1 Diagnoser model 

Definition 2: The proposed diagnoser model structure for the system normal behavior (Q N , 
Mo N ) is an IPN (Q d , Mo d ) where the set of places is P d ={pd] and the set of transitions is T d =T N , 
the incidence matrix C d of (Q d , Mo d ) is the following 

C d =BT<pUCK (2) 

where B T is a qxl vector (q is the number of measurable places of ((Q N ,Mo N ))), qF is the 

output matrix of (Q N , M N ), C N is the incidence matrix of (Q N , M N ). 

The initial marking of the diagnoser model structure for one place is computed as: 

<=SV(<) (3) 

We propose a matrix B that is computed as follows: 

Algorithm 1: Building B 

Inputs: C-incidence matrix of an IPN, 

1 - number of places in the diagnoser-model, 

q - number of measurable places in the IPN, 
Outputs: A matrix B 

1. The "base number" b should be computed. In this case b=2max(abs(cij))+l, where Qj is an 
element of incidence matrix C. 

2. Define a q x 1 vector. 

3. [b° b l ••• b q ~ l ] 

This procedure computes matrix B. 

According to the way in which B was constructed, all columns of C d must different from 

zero and different from each other. 

If a transition U e T - (T R u T F ) is fired in (Q,Mo) then it is fired in (Q d , Mo d ) (it is possible 

since these transitions are event detectable, thus the output system information is enough to 

detect when one of these transition is fired). 

If a transition fyeT R is enabled in (Qd, Mo d ) and A(tj) is activated in (Q,Mo), then tj must be 

fired in (Q d , Mo d ). Thus, if tj is not fired in (Q,Mo), then (Q,Mo) reached a faulty marking. In 

this case the output of the system and the output of the diagnoser are different from each 

other. 

4.2 Error computation and fault isolation 

Definition 3. Error computation. The k-th error is computed by the following equation: 

e k =M k d - B?((pM k ) (5) 

Notice that e k is computed from the diagnoser-model output and not from the marking M*. 
It means that the proposed diagnoser is using the system output and not internal system 
signals (those signals that are non measurable). 

Definition 4. Fault isolation. When e k ^ 0, an error is detected, then a faulty marking was 
reached. The mechanism used to find out the faulty marking is named fault isolation. This 
work proposes the following algorithm to accomplish this task. 
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Algorithm 2: Fault isolation 



Inputs: Mk , Mk d , e k . 

Outputs: p (faulty place), M/(faulty marking) 

Constants: C d is the IPN diagnoser structure incidence matrix 

i = index of the column of C d such that G*(l,i)= ey, 

-\/pe.U,M k (p)=0 
-\/pet r ,M k (p)=0 

- \/p?e ('ti) " fW, M k (p F )=1 

- Mf=Mk 

- Return {p, Mf) 



Example 3. Consider the system of the example 1. The IPN depicted in figure 4 represents 
the behavior of the system of example 1. Since this IPN is input-output diagnosable by 
theorem 1, then a diagnoser can be built for this system. In this case we will use the reduced 
structure presented in this section. 
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The base obtained to compute B is b=2*l+l=3; since we build B using algorithm 9. We obtain 
the following vector: 



B T = [l 3 9 27f 



Therefore C d is: 



C d =[-1 27 1 9 -9 -27 3 -3] 
Hence, its associated IPN is depicted in figure 7. 




Fig. 7. The IPN reduced diagnoser-model 
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The initial marking of the diagnoser is Mo d =[3]. In order to show how the diagnoser works, 
assume that the following sequence tih is executed into the system, then this sequence is 
fired in the diagnoser. Thus the system output is ((p(M k )) T =[l 1 l], and the marking of 

the IPN diagnoser is Mf =[3 1], Then e k = M k d - (B T )((pM k ) = [31] - [31] = 0, thus the system is 
in a normal state. Now if the faulty transition tg is fired, then pu is marked, however no 
change in the output system is detected. If the symbol of ti (A(ti)=a) is given as input to the 
IPN of the system model and IPN diagnoser, then the diagnoser evolves, and M k +i d =[30]. 
Then e k = -1 indicating the existence of an error. The fault isolation algorithm (algorithm 2) 
detects that the column 1 of C d is equal to e k , thus ti was not fired in the system. Then the 
same algorithm detects the faulty marking and determines that the faulty place pu is 
marked. 

5. Distributed diagnoser 

5.1 Model distribution 

In order to build a distributed diagnoser, the IPN model (Q, Mo) can be conveniently 
decomposed into m interacting modules where different modules share nodes (transitions 
and/ or places). 

Definition 5. Let (Q,Mo) be an IPN. A module ju k = (Nk, 2k, ha, 9k) is an IPN subnet of the 
global model (Q,Mo), where: 

• N k = (T k , P k/ I k/ O k/ If, Of, M ok ) where: 

• T.czT, 

• P k = P k L uP k c ; P k L cP; Pf represents the communication places among modules', this set 
is a copy of some places Pi L that belongs to other modules, l?k. P k c is the minimal places that 
is required for the transitions of the module are event-detectable. M(P k c )= M(Pi L ). 

• I k (O k ): Pf xT k ^ Z + , s.t, I k (pi,tj) = I(pi,tj) (O k (pi,tj) = 0(pi,tj)), Vpi e Pf and Vtj e T k . 

• If (Of): Pf x 7W Z\ s.t, If(pi,tj) = I(pi,tj) (Of(pi,tj) = Ofaty), Vpi e Pf and Vtj e 
Ti, l?k. If (Of) are the input(output) arcs to from piePf to transitions of other modules. 

• M ok = M \pk 

• I k = {aeS | 3U, U eT k/ X(U) = a} 

• A k : T k -^ I k u {s}, s.t. A k (U) = X(U) and U eT k 

• epic : R(jU k , Moic) — > (Z + )^ fc , q k is restricted to the outputs associated to P k . 

Definition 6. Let (Q,Mo) be an IPN. A distribution DNj of (Q,Mo) is a finite set of m modules, 
i.e., DNi={jUi, jU2,.. .,ju m ). The distribution DNj holds the following conditions: 

1. |Xi^/ = fcULP k L = P ( Tt is a partition over Pf). 

2. Ur=i T k = T (It is not a partition) 

The set of communication places P com = \J" =l P k c of a distribution represents the measurable 

places of each module ju k e DNj needed to guarantee the event-detectability property of ju k . 
Definition 7. Let DNj be a distribution of (Q,Mo) and a=Utj. . .t k . . . be a sequence of £(Q,Mo). We 
define the natural projection PT k of £(Q,Mo) over the languages of the modules ju k e DNj of 
the following way: 

PT k :£(Q,Mo)^%k,M k) 

\Ht....tt g £(Q,Mo) 
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PT k (s) = e 
PT k {t,t r ..t s t q )-. 



PT k {t,t r ..t s ) ift q £T k 
PT t (tt r ..t s )t q ift g eT k 



There exists the input-output symbol projection over the input and output module symbols. 
Let G) = {ao,yo){ai,y-i)...{a n ,y^) be a sequence of symbols of (Q,Mo) 
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where y t (s) = if the measurable place does not belong to ju^ 

where y i (s) = y i (s) otherwise. 

Example 4. Consider the IPN system model depicted in the figure 4. We partition the 
producer-consumer model to obtain a distributed model. The figure 8 depictes a distribution 
DN{. For the sake of simplicity, we use in the example the same names for duplicated nodes 
(places or transitions) belonging to different modules. The distribution has three modules, 
i.e., | DNj | = 3; h c (Oif) is represented by the dashed arcs. The module jui has the transitions 
Ti = {t^tzt^tg} and the place set Pi = {pi,p2,p3,pu}^{pio}- We are preserving the property of 
event detectability of common transitions duplicating measurable places, and using these 
copies in different modules. 




Module 1 




Module 2 




Module 3 



Fig. 8. Distributed Interpreted Petri Net Models 

5.2 Distributed input-output diagnosability 

The results of centralized diagnosability are applied to the modules. 

A module is locally diagnosable if, for every local fault we can detect it only using local 

information, else it is conditionally diagnosable. 

Definition 8. (Local Diagnosability) A module ju n e DNj is said to be locally input-output 

diagnosable in k < oo steps if any faulty marking Mjh e R(ju n , Mo n ) is distinguishable from any 

other Mkn e K(// n , Mo n ) using words a n , where PA t (&>) = o) n . 



82 



Advances in Robotics, Automation and Control 



Definition 9. (Conditional Diagnosability) A module ju n e DNi is said to be conditional input- 
output diagnosable in k < go steps if any faulty marking Mjh e R(ju n , Mo n ) is distinguishable 
from any other Mkn e R(Mn, Mon) using words co n and co m , where PA^(co) = co n and 

PA m (co) = co m , co m denotes the set of all input-output sequences that lead to a marking a 

duplicate place in ju m , where ju n ^ ju m . 

5.3 Communication channels 

Definition 10. The border transitions between two modules juk and ju z is: T^horder = {U I U e {Tk 
n T z }}. This concept can be extended to several modules. 

The communication channels between two modules are represented by h c (Ok c ). We assume 
that every module can communicate with every other module. The firing of a transition tj e 
Tk may be local to the module juk and cause only a local marking change, or it may involve 
communication with another module if tj e T^border- Every time a transition tj e T^border is 
fired in juk then a message is sent to every ju z containing the same tj e T^border to put a token in 
some place p x e P z c such that p x e 'tj. In (Lampson, 1993) it is proposed different ways to 
create protocols for implementing reliable messages. 



5.4 Redundant diagnoser 

Since distributed diagnosers leads to the use of several computers (CPU), then, redundancy 
can be introduced in the diagnosers. For instance, the Triple Modular Redundancy (TMR) 
can be used in this case. 

Assume that a distribution DNi={jui, jU2,...,ju m } was obtained and that it was distributed over 
m computers (see figure 9). 

System 
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Fig. 9. On-line Distributed Diagnoser 

Then, a TRM scheme can be also applied to figure 9 (see figure 10), increasing the diagnoser 
reliability. 
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6. Conclusions 

This chapter introduced the diagnosability property in DES modeled using IPN. It presented 

a structural characterization of this property using the T-semiflows of the IPN. 

The approach herein presented exploits the IPN structure to determine when it is 

diagnosable, this approach leads to polynomial characterization of diagnosability. 

Based on the DES model, three different types of diagnosers were presented. The first one 

was a centralized version, allowing to detect and locate faults. Sometimes, however, the 

system could very large; leading to large diagnoser models thus the other two diagnosers 

are designed to tackle this problem. 

The second diagnoser is a reduced scheme. It uses one place; however the number of tokens 

could be large. The third diagnoser is a distributed one, were the diagnoser model is 

distributed over different computers. Adopting this approach the problems that appear in 

centralized versions are eliminated. Moreover, in this last case, a redundant diagnoser can 

be used for increasing the reliability of the distributed diagnosers. 
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1. Introduction 

The decision theory studying control problems for systems of various natures (technical, 
biological, socio-economic) has become independent science branch during last decades. 
The decision theory actively uses methods of mathematics, psychology and informatics. The 
cognitive simulation is one of the new approaches of modern decision theory. The main 
purpose of this chapter is forming complete idea of cognitive approach actively developed 
now in the control science and interdisciplinary sciences (sociology, economy, etc.), further 
specification of basic concepts of this approach in the control science, as well as defining a 
class of control problems that are advisable to be solved by the cognitive simulation 
(simulation on the base of cognitive map). 

The cognitive approach to simulation is directed to development of formal models and 
methods that support the intellectual process of problem solving owing to taking into 
account the cognitive abilities (perception, representation, cognition, understanding, 
explanation) of control agents. The basic technique of ill-structured problem solving is 
structurization of knowledge about object with further its environment and construction of 
a cognitive map (the static model of situation) and a dynamic model. The technique includes 
monitoring of dynamic of factors of the model (their tendencies), analysis of the model 
structure with the use of SWOT-approach, and modeling that permits to determine and 
solve ill-structured problems. This technique allows supporting of a vital control task that 
consists in goal setting of situation development, as far as solution of discovered problems 
turns into the system development control task. This technique application is useful when 
designing a strategy of development of social and economic objects. 

The problem of risks for the results validity that arise due to the human factor in the 
cognitive approach is considered and two kinds of risks which we can pertinently treat as 
cognitive risks are exposed. On the example of a real cognitive map of a complex and ill- 
structured situation the analysis of some cognitive risks concerned with causal influence 
transitivity is carried out. Some explanatory mechanisms and criteria for the early detection 
of such risks are proposed. The issues important for further development of the cognitive 
approach to decision-making in the ill-structured situation control and especially of causal 
mapping techniques are highlighted. 
The chapter includes the following topics: 

1. Brief review of cognitive approach in control: evolution and some trends in its 
development. The problem of risks due to the human factor. 
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2. The models and methods of ill-structured problem solving on the basic construction 
and analysis of the cognitive map. Presenting the risks related with formalization of 
initial expert knowledge about ill-structured problem situation, by the example of 
cognitive map construction. 

3. Analysis the experience of cognitive techniques application to control of socio-economic 
development and some trends of further development of cognitive approach. 

2. Brief review of cognitive approach in control: evolution and some trends in 
its development. The problem of risks due to the human factor 

2.1 Brief history of cognitive approach evolution 

Origins of concept "cognitive map" lie in psychology (Tolman, 1946). Studying cognitive 
maps - subjective representations of spatial organization of outer world - has gained in 
fundamental importance in the framework of studying features of human perception of his 
surroundings. The cognitive map is the concept concerning cognitive processes related to 
gathering, representation, and processing of information on the environment during which 
an individual is not only a passive observer, but actively interacts with his environment. 
Forming cognitive maps by an individual is understood as the process consisting of series of 
psychological transformations. By means of these transformations, an individual gathers, 
stores, copies, recalls and manipulates information on relative locations and attributes of his 
spatial surroundings. This process is essential component of decision making for spatial 
behaviour. The psychological research is directed to a greater extent toward studying these 
very processes and their influence onto forming certain representations allowing an 
individual to act and make decisions in his environment. 

In political science and sociology, the method of cognitive simulation was developed in 
1960-1980 in papers of American researcher R. Axelrod and his colleagues from USA and 
Scandinavia (Heradstvein & Narvesen, 1978; Axelrod, 1976). 

It should be noted that many authors also use the term "cognitive mapping". In any case, 
research of problem situation is based on model construction on the base of a cognitive map. 
The differences consist only in applied modifications of cognitive maps and methods of 
their formal processing. 

In political science and sociology, "cognitive map" is not related to spatial orientation. It is 
interpreted as an individual's schematic representation of world image fragment relating to 
concrete problem situation. In this context, cognitive map is the way for representing 
thought structures directed toward a concrete problem and allowing to simulate politician's 
mentation during deliberation of action stimulating identification of further events 
(Heradstvein & Narvesen, 1978). 

Construction and analysis of cognitive maps allow revealing causal structure of reasoning 
presented in political texts and, on this base, making conclusion on text author's vision of 
political situation, defining factors taken into account by politicians while making decisions. 
Axelrod developed the method of cognitive simulation relying on ideas of psycho-logic 
(Abelson & Rosenberg, 1958), causal deduction (Shapiro & Bonham, 1973), as well as graph 
and decision theories (Axelrod, 1976). As the main problems of decision making, he 
distinguished the problems of situation explanation and test of hypothesis on situation 
arrangement, as well as prediction problems and problems of decision selection from a 
number of alternatives. On the base of these studies it was shown that in complex situations 
an individual is inclined to simplify situation representation, to not notice feedbacks, etc. As 
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a consequence, long-term effects, interrelations between different problems, etc., are not 
taken into account while making decisions. 

Axelrod placed the models constructed on the base of cognitive maps among normative 
models in that sense that they organize cognitive activity of an individual during the 
process of decision making. Just as any formalization, the cognitive map and methods of its 
analysis prescribe an individual his way of making decisions in complex situations. 
The methods of cognitive simulation have found an application in collective working out 
and making decisions (Hart, 1976; Roberts, 1976). English scientist K. Eden developed 
general approach to construction of cognitive maps (Eden, 1988), resting on research in the 
field of decision making psychology, in particular, on the theory of personal constructs by 
Kelly (1955). Eden outlined importance of Kelly's statement to the effect that effectiveness of 
interaction in group of individuals making decisions essentially depends on each member's 
understanding of ways for situation interpretation by other group members. 
Empirical studies of some authors (Eden, 1988; Axelrod, 1976; Hart, 1976; Holsti, 1976; 
Roberts, 1976; Shapiro & Bonham, 1973) shown that application of cognitive simulation 
methods allows increasing effectiveness of decision making in problem situation. Analyzing 
his own and other's cognitive maps, an individual can define problem representation more 
exactly, find contradictions, understands other individuals. In addition, the cognitive map is 
the convenient tool for changing settled stereotypes contributing to generation of new 
viewpoints. 

At the same time, Axelrod notes a lack of formal methods for construction of cognitive maps 
directed toward reliability and interpretability of results of problem situation analysis 
(Axelrod, 1976). 

As a rule, application of cognitive simulation methods in sociology and political science is 
aimed to revealing representations of an individual making decisions in various situations, 
resolution of subjective conflicts due to difference between problem representations and 
lack of mutual understanding among interacting individuals. 

The development of cognitive simulation methods is substantially conditioned by need in 
analysis of ill-structured systems and situations including multitude of elements with 
different nature, as well as relationships between elements having both quantitative and 
qualitative nature. The concept " ill-structured problem (situation)" was introduced by 
G. Simon (1973). 

The cognitive approach to analysis of ill-structured situations was proposed by Axelrod, 
(1976) and Roberts (1976). Axelrod to a greater extent was engaged into development of 
methodology, while Roberts - into development of mathematical tool. The primary 
precondition for rising this approach consists in boundedness of accurate model 
applicability to ill-structured system behaviour analysis, as well as making control decisions 
on resolution of problem situations. 

For this approach, the subjective understanding and individual's representation of the 
controlled system parameters and relationships between them in form of cognitive map 
consists foundation for construction of ill-structured model of system or situation. 
Subsequent choice of formal methods for processing of representations reflected in the 
cognitive map depends on goals of analysis, as well as characteristic features of ill- 
structured system or situation. 

Nowadays, the cognitive approach to analysis of ill-structured systems and situations is 
actively developed in Russia and abroad (Avdeeva et al., 2007; Kuznetsov et al., 2006; Kulba 
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et al, 2004; Avdeeva et al, 2003; Chaib-draa, 2002; Maximov, 2001; Maximov & 
Kornoushenko, 2001; Kim, 2000; Huff, 1990; Kosko, 1986; Sawaragi et al, 1986; Heradstvein 
& Narvesen, 1978; Roberts, 1976). One of the typical trends of this development consists in 
looking for mechanisms joining various scientific areas of researching decision problems for 
control of ill-structured systems and situations. 
The distinctive feature of cognitive approach consists in the following: 

process of solving of control problems is considered as cognitive human activity 
including application of formal models and methods as a part or stage of solution; 
another important stage is formalization of representation of ill-structured systems and 
situations, goals, interests and motivations of individuals involved into the problem 
solution processes. 

2.2 Trends in development of the cognitive approach in decision-making 

In theoretical researches on methods for searching and making decisions, such terms as 
cognitive approach, cognitive researches, cognitive modeling or cognitive mapping of 
complex objects, problems, and situations are used more and more often. It can explained by 
growing understanding of inevitable participation of people with their cognitive resources 
(and also restrictions) in the decision of practical problems of control, especially in case of 
complex and ill-structured situations (Abramova & Kovriga, 2008). 

Recently the selective analysis of works proposed at International Conference " Cognitive 
analysis and situations evolution contror (CASC) has been carried out (Abramova, 2007b), 
with the following similar analysis of other publications using term "cognitive" in the 
context of solving the applied control problems. The conference has been held at the 
Institute of Control Sciences of Russian Academy of Sciences for the latter seven years, being 
focused on the integration of formal and cognitive (using resources of the person) problem 
solving methods. 

The analysis has allowed seeing considerable distinction in understanding of what terms 
"cognitive approach" and "cognitive modeling" mean, in how the term "cognitive" 
operates, in scientific and applied problems being solved, in the formalization level, in 
considered cognitive aspects, in involved knowledge of the cognitive science. At all 
distinctions, it is possible to identify two basic directions in accordance with understanding 
of the cognitive approach in narrow and wide sense (in the context of decision-making and 
ill-structured situations). 

Two directions of the cognitive approach. Today, the direction of the cognitive approach 
corresponding to narrow sense of the term is the most developed one. It means that various 
models of cognitive maps are applied as models for representation and creation of 
knowledge about ill-structured situations,. The most conservative branch of this direction 
focused on formal methods, at all does not consider specificity of human factors and 
features of structurization by the person of difficult situations et all; so the word "cognitive" 
bears purely nominative function of a label for models applied. 

However as a whole it is relevant to speak about two trends in development of this 
direction. On the one hand, the positive tendency to larger account of such human- 
dependent stages as formalization of primary knowledge and representations of a problem 
situation, targets definition, etc is observed. On the other hand, the accepted models of 
knowledge and activity of the people solving practical problems (experts, analysts, decision- 
makers) are normative in relation to those people, and the justification of the models is 
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defined by theorists at the level of common sense and traditions. Any knowledge of how 
people really think and what knowledge the cognitive science in this respect has, today are 
not usually involved today. 

The cognitive approach in wide sense is not limited by the choice of cognitive maps as 
models for representing knowledges about complex objects and situations (Abramova & 
Novikov, 2006). The accent initially is made on human-dependent stages. Basically, the 
approach covers a wide spectrum of the models applied in decision-making for ill- 
structured problem situations. However, today the same tendencies in this direction 
dominates, as in the previous one. 

Review (Abramova, 2007b) represents perspectives of more advanced development of the 
cognitive approach, with integration of formal methods and knowledge of psychology and 
the cognitive science as a whole. However, today bridging the gap appears to be difficult, at 
least, because of distinction in scientific languages. Among few exceptions it is useful to 
mention (Obal et al, 2007; Gabriel & Nyshadham, 2008; Vavilov, 2006). 

2.3 The problem of risks due to the human factor 

The problem of risks due to the human factor in the field of formal methods for searching 
and making decisions to control complex and ill-structured situations essentially consists in 
that because of inevitable and substantial human beings' participation in solving practical 
problems (at least, for formalization of primary representations) formal methods basically 
cannot provide validity of received decisions. Note that validity of method application 
results is understood here in wide intuitive sense as capability to rely upon these results in 
solving a specific practical problem. It is also possible to speak about validity of a method as 
its capability to yield valid results. Simplier speaking, such methods (which we refer to as 
subjective-formal ones) are basically risky concerning validity of their results. 
The pragmatical importance of the given problem of risks obviously depends on how much 
significant are risks obtaining invalid results in solving practical problems. By present time 
theoretical, experimental and even practical knowledge is accumulated, leading to under- 
standing or directly saying that human factors can be the significant source of risk for 
quality of results (Abramova, 2007a). 

The most impressing is the research on "logic of failure", presented in D. Dbrner's known 
book (Dbrner, 1997). By means of vast computer-based simulation of how real people solve 
problems of the complex and ill- structured dynamic situation control, a number of typical 
errors is revealed which are inherent not only in dilettantes but also in experts at work with 
complex situations. These results lead to conclusion that some of the above errors should be 
expected for any kinds of cognitive maps whereas others should be characteristic of 
dynamic map modeling. Strictly speaking, in Dorner's experiments it is possible to admit 
that the risks of errors stem not only from natural ways of thinking but also from the 
accepted model for representation of knowledge about situations in the computer. However, 
Dbrner finds evidences in favor of his theory in the analysis of known natural-thinking 
decisions concerning Chernobyl. 

One more class of risk sources is revealed by O. Larichev (Larichev & Moshkovich, 1997). 
with his school in the traditional formal decision theory. It refers to methods and operations 
of receiving initial information from decision-makers (in other terms, knowledge 
acquisition) for subsequent processing with formal methods. Stemming in the accumulated 
psychological knowledge of more than 30 years, the inference has been drawn that "soft" 
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qualitative measurements such as comparison, reference to a class, ordering are much more 
reliable, than appointment of subjective probabilities, quantitative estimates of criteria 
importance, weights, usefulnesses, etc. To increase reliability of such operations (in our 
terms, to decrease risk of human errors) the idea of psychologically correct (in other terms, 
cognitively valid) operations has been advanced by the school. From the above results it 
follows that validity of cognitive modeling should be various for different kinds of cognitive 
maps depending on kinds of estimates demanded from experts. 

There is also a wide spectrum of psychological researches in the field of the limited 
rationality of the person, not relating to solving control problem activities, which evidencing 
to numerous types of risks in intellectual activity of a person. 

To add evidences of practical significance of cognitive risk factors in decision-making, it is 
relevant to mention that some significant risk factors have been found out by these authors 
in the applied activity on safety-related software quality assurance including formal-method 
quality estimation and control. This practice along with further theoretical analysis have 
considerably expanded representations of the risk spectrum produced by theorists 
developing or choosing models for knowledge representation and activity of people solving 
practical problems, in comparison with the risks exposed by Larichev's school (Abramova, 
2004; Abramova, 2006). For example, it has appeared that application of very natural, from 
the mathematical point of view, method of linear convolution of normalized estimates on 
partial quality indicators for estimation of a complex indicator creates paradoxical risk of 
loss of controllability on partial indicators. 

Moreover, the analysis has confirmed presence of some general cognitive mechanisms of 
risk for diverse models of subject-matter experts 7 knowledge and related formal methods 
what serves evidence of reasonability of the general approach to the problem of risks due to 
the human factor in decision-making. 

The subjective aspect of the problem of risks is that even more or less widely spread peaces 
of knowledge relative to risks and their sources mainly are not noticed by scientific 
community or, at the best, are underestimated. This ignorance is quite explainable 
theoretically with taking into account psychological risk factors (with the models presented 
in (Abramova, 2006; Abramova, 2007a). Thereby, there is a soil for theoretical development 
and practical application of subjective-formal methods, which cannot provide decision 
quality (adequacy, validity, reliability, safety) comprehensible for critical domains. 
Among few works in the field of cognitive mapping, with recognizing not only the human 
factor influence, but also necessity of researches in this direction, it is worthwhile to note 
(Bouzdine-Chameeva, 2006), where validity of cognitive maps with internal validity 
between the data and the conceptualization of the data, including the definitions of concepts 
and influences, has been discussed, proceeding from the general ideas of content-analysis 
reliability. 

Cognitive risks. Two kinds of risk factors.In diversity of risks due to the human factor in 
the considered field the special class is formed by cognitive risks which are explainable with 
taking into account factors (mechanisms) concerning cognitive sphere of a person. All risks 
mentioned above are referred to cognitive risks as well as a number of others, which are 
reported about in publications and private communications without doing them subject of 
the scientific analysis. 

First of all, researchers are interested in the factors that, on the one hand, have regular 
nature (i.e. do not concern to dysfunctional cognition), and on the other hand, are hardly 
explainable from positions of common sense or even contradict it; so usually they are not 
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assumed by the mathematicians developing working out normative models for expert 
knowledge representation or ignored if known. 

Earlier cognitive processes have been analyzed in which subjects of intellectual activity 
(analysts, experts, decision makers and other staff who participates in searching and making 
decisions at collaboration with computers) turn to be under the influence of " ambient 
intelligence' 7 due to imposed forms of thinking (Abramova & Kovriga, 2006; Abramova, 
2006; Abramova, 2007a). This influence leads to dependence of decisions on theoretical 
beliefs of specialists on formal methods and computer-aided decision support systems and 
technologies. Therefore, results in risks of invalid decisions. Two kinds of risk factors 
explainable with the suggested models have been exposed which are pertinently considered 
as cognitive risks. 

The risk factors psychologically influencing validity of expert methods during their 
application by experts belong the first-kind factors, or factors of direct action. Such factors 
can either objectively promote invalidity of results, or raise subjective confidence of experts 
in objective validity for the method application outcomes. One can tell that the latter 
represent themselves as factors of belief. Agents of these factors of influence are experts; just 
they appear in conditions which that may lead, eventually, to insufficiently valid (in the 
objective relation) outcomes. 

The second-kind risk factors or factors of an indirect action psychologically influence upon 
validity of expert methods during their creation and justification. The agents of influence of 
such factors are the creators of methods, scientists and experts producing standards who, in 
turn, are subject to influence of scientific norms, paradigms, etc., that is the strongest factors 
of belief. Typical examples of the first-kind risk factors are the natural mechanisms of 
thinking with risk of errors, as in the case of the errors discovered by Dorner. Typical 
second-kind risk factors are psychologically incorrect models of knowledge of the experts, 
creating risk of unreliable (unstable or inconsistent) data from experts, with the models 
being supported by belief (often unconscious) in their validity. 

Amongst a number of cognitive risk factors having been found out by these authors both in 
theoretical and experimental researches, as well as in practice, there are risks related to 
belief in universality of the principle of pairwise preference transitivity in decision-making 
(Abramova & Kovriga, 2006 ; Abramova & Korenyushkin, 2006). The principle means that 
from a y b ("a is preferable over b") and b y c it always follows a y c , though rationally 
reasoned violations of this principle are known (for example, preferences based on 
multicriteria comparisons). 

From now onwards, the given work concerns one family of cognitive risks having been 
discovered quite recently. Risks take place in cognitive mapping based on causal maps. 
They are related to the property of causal influence transitivity and its violations. They have 
been admitted as a hypothesis by analogy to known rationally reasoned violations of 
transitivity of pairwise preferences and later found out in real causal maps. 

3. Some methods of ill-structured problem solving on the basic a cognitive 
map. Some risks concerned with causal influence transitivity 

For further turning to the cognitive approach in the narrow sense, it is necessary to clarify 
the concept of cognitive map in the meanings used in the decision-making context. 
In this field of knowledge the concept of cognitive map is used more and more widely 
beginning from (Axelrod, 1976), but it takes various meanings, without saying about 
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essentially differing concept of cognitive map in psychology. Recent years have brought out 
a number of reviews and articles with extensive reviews in which the diverse types of 
cognitive maps and other concept maps are compared, differing in substantial and formal 
interpretation, as well as in sights at their role in searching and making decision processes 
and control of complex objects and situations (in particular, Eden, 1988; Kremer, 1994 ; Mis, 
2006;. Kuznetsov et al, 2006; Bouzdine-Chameeva, 2006; Abramova, 2007b; Avdeeva et al, 
2007; PenaetaL, 2007). 

In this work term " cognitive map" refers to the family of models representing structure of 
causal (or, that is the same, cause-effect) influences of a mapped situation. Formally, the 
obligatory base of all models of the family is a directed graph, the nodes of which are 
associated with factors (or concepts) and arches are interpreted as direct causal influences 
(or causal relations, connections, links) between factors (Kuznetsov, Kilinich & Markovskii, 
2006). Usually the obligatory base (that is the cognitive map in the narrow sense) is added 
with some parameters, such as an influence sign ("+" or "-") or influence intensity, and some 
or other interpretations both substantial, and mathematical are given to the map (Fig. 1). 




Fig. 1. Example of cognitive map 

Various interpretations of nodes, arcs and weights on the arcs, as well as various functions 
defining influence of relations onto factors result in different modifications of cognitive 
maps and formal means for their analysis (Kuznetsov, Kilinich & Markovskii, 2006). Owing 
to multitude of cognitive map modifications, one can distinguish different types of models 
based on cognitive maps (in short, cognitive map models). Models of this family that are 
often referred to as causal maps or influence diagrams cover a wide spectrum of known 
types of models for cognitive mapping. 
By type of used relationships, five types of cognitive maps are distinguished (Huff, 1990): 

1. Maps that assess attention, association and importance of concepts: With these maps, 
the map maker searches for frequent use of related concepts as indicators of the 
strategic emphasis of a particular decision maker or organization, for example, and look 
for the association of these concepts with others to infer mental connection between 
important strategic themes. He also can make judgments about the complexity of these 
relationships or differences in the use of concepts; 

2. Maps that show dimension of categories and cognitive taxonomies: The map maker 
investigates here more complex relationships between concepts. He can dichotomize 
concepts and construct hierarchical relationships among broad concepts and more 
specific subcategories. The maps of this type have been used to define the competitive 
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environment, and to explore the range and nature of choices perceived by decisions 
makers in a given setting; 

3. Maps that show influence, causality and system dynamics (causal maps): These maps 
allow the map maker to focus on action, for instance, how the respondent explains the 
current situation in terms of previous events, and what changes are expected in the 
future. This kind of cognitive map is currently, has been, and is still, the most popular 
mapping method; 

4. Maps that show the structure of argument and conclusion: This type of map attempts to 
show the logic behind conclusions and decisions to act. The map maker includes here 
causal beliefs, but looks more broadly at the text, as a whole, to show the cumulative 
impact of varied evidence and the links between longer chains of reasoning; 

5. Maps that specify frames and perceptual codes: This approach supposes that cognition 
is highly conditioned by previous experience, and that experience is stored in memory 
as a set of structured expectations. 

Applied practice shows that the maps of the third type are expedient for studying of ill- 
structured systems. Choice of method for structuring of ill-structured systems and situations 
in a form of set of factors and causal relations are conditioned by that the events and 
processes of operation and development of ill-structured systems include various events 
and trends defined by many factors, at that each of them in turn has an influence on some 
number of other factors. The networks of causal relations between them are formed 
(Kuznetsov et al., 2006; Dorner, 1997). Book of well-known German psychologist Dorner 
(1997) devoted to studying thought on control subject and analysis of causes of mistakes 
while resolving problem situations in operation and development of complex systems 
indicates that a momentary situation with its features is only actual state of system and its 
variables. One should not only understand events that happen, but also foresee events that 
will happen or may happen in the future, as well as suppose a way for situation changing 
depend on concrete interference. It requires structured knowledge, i.e. the knowledge of 
relationships and mutual influences of system variables. Dorner notes that in ideal case this 
knowledge is represented in the form of "mathematical functions", but in case when the 
latter cannot be constructed one can apply diagrams of causal relations allowing to 
reconstruct various assumptions (hypotheses) in mind of the subject of control, at that not in 
the form of "causal chains", but in the form of "causal networks". 

According to Kuznetsov, Kilinich & Markovskii (2006), the situation analysis problems on 
the base of cognitive maps can be divided into two types - static and dynamic. Static 
analysis or influence analysis is analysis of considered situation via studying structure of 
mutual influence in a cognitive map. The influence analysis chooses factors with the 
strongest influence onto goal factors that are the factors with values to be changed. The 
dynamic analysis lies in the base of generating possible scenarios of situation development 
with time. Thus, abilities of solution of analysis and control problems are defined by type of 
used models, static or dynamic. 

As a rule, two types of mathematical tools are used for carrying out these kinds of analysis - 
linear dynamic systems and fuzzy mathematics. 

Nowadays, the research abroad is mainly devoted to the development of models on the base 
of fuzzy mathematics and static models using different mathematical tools (Chaib-draa, 
2002; Kim, 2000; Kosko, 1986; Sawaragi et al, 1986; Heradstvein & Narvesen, 1978). 
In Russia, along with the development of methods for static analysis of cognitive maps, 
special efforts are directed to research of ill-structured systems and situations using linear 
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dynamic models that are presented in this paper by works of Russian scientists (Avdeeva et 

al, 2007; Avdeeva, 2006; Kulba et al, 2004; Avdeeva et al, 2003; Maximov & Kornoushenko, 

2001; Maximov, 2001). 

In linear dynamic model based on cognitive map, factor is formally defined as variable 

taking values from some numerical scale. 

The model construction is based on the cognitive structurization of knowledge about object 

and its environment. 

The purpose of structuring consists in revealing of the most essential factors describe a 

"boundary" layer of interaction between object and external environment, as well as 

establishing of qualitative relationships between cause and effect between them. 

The cognitive structurization is finished by formalization of singled out knowledge that 

consists in generalization of essential information into a system of basic factors. 

Analysis of a graph model of a situation associated with a cognitive map allows to reveal the 

structural properties of a situation. The basis of the model is a weighed digraph G =(X, A), 

where X is a set of nodes that biuniquely corresponds to the set of basic factors, A is a set of 

arcs reflecting the fact of direct influence of factors. Each arc connecting some factor xi with 

some factor Xj has the weight aij with the sign depicting the sign of influence of the factor xi 

on the factor Xj, and the absolute value of a^ depicting the strength of the influence. Thus, the 

cognitive map can be examined as the connectivity matrix A g of the graph G. 

When constructing a cognitive map , the set of basic factors, X, is grouped in blocks relevant 

to external environment, X ext , and an internal environment, X int = X\X ext . Besides 

determination of factors and influence between them the vector of initial factor trends, 

Xext(0) u X in t(0), is established. 

Change of factor value with time is given by Kulba et al. (2004), Maximov (2001), Maximov 

& Kornoushenko (2001) 

x i (t + l) = x i (t) + Xa ij (x j (t)-x j (t-l)), i = l,...,N (1) 

where Xi(t+1) and Xi(t) are the values of i-th factor at instants t+1 and t, respectively, 
Xj(t)-Xj(t-1) = AXj(t) is the increment of factor Xj at instant t characterizing the rate of 

change (trend) of Xj, a% is the weight of the factor Xj influence onto factor xi, Ii is the set 
consisting of numbers of factors directly effecting the factor xi. 

With knowledge of the initial situation state X (0) and accepting x (t) =0 for all t <0, the state 
of a situation in self-development (without applied control) at any moment t can be 
characterized by the vector of factor values 

X(t) = (E N + A+ A2+ . . . + At) X(0) (2) 

Estimation of the sum of this series can be obtained only if the graph G adjacency matrix A g 
is stable. Then all elements of this series approach to finite limits at unlimited increase of t. 
To determine the transitive closure, it is sufficient to consider N terms in a power series of 
matrix B, where N is the order of matrix B, i.e. the number of basic factors in the situation 
cognitive map. Then the transitive closure of the matrix B is estimated by the matrix 



Q = E N +A + A 2 +... + A N =(E N -A)~ 1 (3) 
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Matrix Q characterizes the direct and mediated influences effecting each factor. 
Let U(0)=(ui(0),...,u p (0)) be the vector of pulse control effecting at the moment £=0 the factors 
X. Then, with knowledge of the initial state of situation X(0), the state of the situation at any 
moment can characterized by the factor value vector 

X(t)=QX(0)+QBU (4) 

where u = (ui,...,u p ) T is the external input vector and B - (0,1) - is the (pxn)-matrix with 
nonzero elements specifying numbers of corrected coordinates of the initial state X (0). 
The dynamic model is constructed to obtain a new knowledge of structure and dynamics of 
considering situations. On the basis of that model, one can carry out the scenario research 
with use of methods of computer modeling of self-development and controlled 
development of ill-structured situation (Maximov, 2001; Maximov & Kornoushenko, 2001; 
Avdeeva, et al, 2003; Makarenko, et al, 2004). 
Our research group focuses special attention on the following: 

1. searching and development of structuring methods aimed at construction of cognitive 
maps; 

2. increasing technological effectiveness of scientific and instrumental support for solving 
practical control problems: 

3. approach to revealing and prohibition of the semantic errors and risks of formalization 
(before application formal method to analysis of ill-structured situation). 

The problem of risks outlined below in section 3.1 represents advanced approach in wide 
sense, with taking cognitive mapping as the representative example. 

On the basis (l)-(4) has developed the following methods and approaches have been developed 
(Avdeeva et al, 2007; Avdeeva, 2006; Avdeeva et al., 2003; Maximov & Kornoushenko, 2001): 
method of structure and goal analysis of ill-structured system development (section 
3.3); 

SWOT-analysis on the basis of analysis of the model structure (section 3.2); 
approach and methods for solving ill-structured problems and deriving scenarios of ill- 
structured system development; (section 3.3); 

approach to studying conflict situations generated by contradictions in interests of 
subjects influencing development of considered system and other. 
At that, the problem is defined as discrepancy between current condition of ill-structured 
system and its dynamic and desired condition that is given by the subject of control. 
Complex application of the mentioned methods allows carrying out static and dynamic 
analysis while studying ill-structured systems. The socio-economic systems (SEO) constitute 
a typical class of ill-structured systems that are expedient for application of cognitive 
simulation for development problem solving. 

But it is necessary to take into account that such methods essentially depend on techniques 
of revealing of the factors and definition of their interrelations describing situations under 
research. Construction of the cognitive map reflecting representation of complex system 
development includes forming of the conceptual scheme of the situation (the description of 
the subject domains defining complex system development); choice or revealing of the 
important factors (detailing of the conceptual scheme); definition of interrelations between 
the factors. 

Despite of popularity of such methods, now there is no coordination in the literature 
concerning a way of revealing of the important factors influencing and defining the 
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situation under research. Methods of a cognitive map construction can be useful as to 
understanding cognitive processes of the persons participating in decision-making, and as a 
basis for control of active research of complex situations (Avdeeva, 2006). The analysis of 
works of scientists (Axelrod, 1976; Hodgkinson, Maule & Bown, 2004; Avdeeva, 2006, and 
other), actively applied cognitive methods for research of complex system, shows, that 

• revealing of factors and interrelations by means of a content-analysis of documents, in 
particular, in the first work, devoted to a cognitive map application for support of 
decision-making, the author results bases of such analysis of stenograms of session of 
politicians; 

• revealing of factors and interrelations by means of the analysis of expert opinions thus 
are involved experts on the subject domains; 

• revealing of factors and interrelations by means of the analysis of quantitative data, for 
example, regression analysis. 

The choice of a method for cognitive map construction depends on availability of data and 
also on the purposes of researchers. The listed above methods are not devoid of disadvantages. 
For example, the main problem in the documentary approach consists in that information is 
represented differently. Thus, the stenograms of debate of English politicians were represented 
as well structured material. Other documentary sources can unlikely be considered as well 
structured. One of the main disadvantages of the expert methods is insufficient validity of 
revealing of factors and their interrelations. The analysis of quantitative data assumes that the 
revealed factors and interrelations are impartial. But to get some factor in sight, the 
quantitative information about it and its influence should be collected. 

On the basis of long-term experience of practical application of cognitive methods in the 
analysis of problem situations by the example of development of social and economic 
objects, our research group has developed techniques of carrying out strategic meetings and 
heuristic schemes of a cognitive map construction on the basis of PEST and SWOT 
(Strengths, Weaknesses, Opportunities, Threats) ideologies. This group of methods is 
directed onto structurization and coordination of primary representations of problem 
situation of non-uniform collective of experts. 

As a whole, application of such techniques is complicated by lack of checking adequacy 
criteria for cognitive models as well as presence of reliability problem for results of 
application of expert methods. 

3.1 Some risks concerned with causal influence transitivity and their analysis. 
The example description. The facts. In fig. 2 the fragment of a real cognitive map slightly 
simplified to demonstrate action of risks is presented. Influence in pair of factors (3,2) at the 
verbal level is interpreted as follows: " increase in access of manufacturers to gas export 
pipelines (with other things being equal) causes increase in volume of extracted gas". This 
influence is positive (in mathematical sense) that means the same direction of changes of 
factors. Positive influence in pair (4,2) is verbalized similarly. Influence in pair (2,1) is 
negative: "increase in volume of extracted gas (with other things being equal) causes 
decrease in deficiency of gas in the country". 

All three influences, as well as the set of factors significant for the investigated situation of 
dynamics of the market of gas in the country, are established by the expert. (Substantially, 
this map corresponds to a situation when there are stocks of gas and manufacturers have 
resources for increase gas production in volume but their access to means for its delivery to 
consumers is limited.) 
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False transitivity 
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Fig. 2. An initial fragment of a cognitive map with false transitivity 

According to formal model of causal influences and intuitive logic, from positive influence 
3 — ^>2 and negative influence 2 — ^-»1 follows transitive negative (in mathematical 
sense) influence 3 — ^-»1 ; influence 4 — ^-»1 is deduced similarly. 

However later the expert has noticed that " logically deduced" influence 3 — ~-^\ is absent 
in reality: thereby false transitivity of influences takes place in the map. The analysis of 
expert knowledge of the situation leads to the following correction (fig. 3). 
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Fig. 3. The corrected fragment of a cognitive map 

It is worth while to underline that at such refinement replacements of influence 3 — + -^7. 
with3^^->2", 4^^->2 with 4^^->2' and 2^-^\ with 2' — >1 , in essence have not 
changed expert's interpretation of influences: only the form of representation of knowledge 
has changed only. However, in this way the chain 3 — + -^7. — ^-»1 generating false 
influence 3 — ~-^\ has disappeared. 

Essentially different situation occurs with introduction of additional negative influence 
2" — ~-^T . This influence means that at increase of access of manufacturers to export gas 
pipelines (with other things being equal) it is possible to increase volume of the gas 
extracted for export not only by means of increase in volume of extraction, but also by 
simple « valve switching». Thus growth of volume of gas for export is made at the expense 
of decrease in volume of gas for home market. In this case knowledge is entered into a map, 
well known to experts, but not represented within the frame of initial system of concepts 
(factors). 
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As a result of correction new transitive influences 3 — —>T, 3 — ^—>1 have appeared. 
Instead of positive (in substantial sense) situation in the map of fig. 1, when it is possible to 
reduce deficiency of gas at the expense of access of manufacturers both to internal, and to 
external gas pipelines, more complicated and more realistic situation comes up in the map 
of fig. 3. 

Along with positive (as a matter of fact) transitive influence 4 — ^-»1 , negative (as a matter 
of fact) influence 3 — j — »1 takes place, and their proportion at the decision of the problem of 
gas deficiency in the country demands comparative estimation of influences. 
The invalidity of the fragment of cognitive map in fig. 2 relative to reality, or in other words, 
error in cognitive mapping is obvious. 

The above practical example along with some other ones serve as actual evidences of the 
fact that in cognitive modeling of complex situations there are possible cases of erroneous 
inferences by transitivity, i.e. false transitivity. 

The analysis of cognitive risks. In the above example the false transitivity can be explained 
with cumulative action of two modes of risk factors in the course of cognitive mapping: 
assumption of causal influence transitivity as the universal principle and disproportion of 
extension of concepts. 

First, in each of the three direct influences between factors of the initial map having been set 
by an expert, there is disproportion of extension of concepts with excess of extension of 
concept 2, which denotes the influence receiver in pairs (3,2), (4,2) and the influence source 
in pair (2,1). 

Denote that in cognitive mapping it is traditional to speak about factors (concepts) as causes 
and effects. However we prefer, at least in the analysis, to speak about sources and receivers 
of influences , because, in modeling of complex and ill-structured situations substantial 
cause-effect interpretations of individual influences in a map quite often happen more or 
less difficult. (There is such a situation in the above example.) Moreover, in the theoretical 
analysis which we spend it is more exact to distinguish "f actor s" and " concepts of factors 7 ' 
(that is concepts designating factors). It is relevant to speak about factors at the analysis of 
situation content, and more pertinently to speak about concepts of factors when it is a 
question of the logic analysis of concept extensions. 

Excess of extension of concepts in some direct influences informally means that it would be 
possible to take concepts with smaller extension for mapping the same substantial cause- 
effect influences. Just this action has been made at correction. 

It is hardly admissible to consider such disproportions with excess of extension of concepts 
as errors because they are typical for the conceptualization of complex and ill-structured 
situations. This is evidenced both with practice of cognitive mapping and with informal 
reasonings of experts on such situations. Therefore we consider such disproportions as 
cognitive risks. They are natural for carrying to first-kind risk factors , the carriers of which 
are experts and which objectively reduce validity of cognitive modeling in complicated 
situations. 

Causal influence transitivity is accepted as the universal principle by theorists. It leads to an 
automatic inference of indirect influences by transitivity at the formal simulation of a given 
map. Thus the given assumption should be considered as the second-kind risk factor. 
Some criteria for early detection of risks. Let us more formally define some criteria which 
could help an expert to monitor risks and make the decision on possibility to correct 
disproportional concept extensions in case of false influence transitivity detection. In the 
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definition, the fact, which has been found out in practice, is taken into account: the same (as 
a matter of fact) causal influence can be represented in a cognitive map in different forms so 
that we speak about different representations of the influence. Let we have factors A, Bi 
(represented with the same name concepts), which are linked by direct causal influence 
Bi— >A, and let there exists (is found by an expert) factor B2 such that replacement of 
representation of influence Bi— >A with B2— >A does not change the influence substantially, 
and herewith 

% x => % 2 > 
where % ,i = 1,2, stands for the extension of the corresponding concept, and the relation 

between extensions is treated as usual inclusion or, that is the same, verbally: "ci has smaller 

extension than C2". Then the factor B2 is more proportional in its concept extension than Bi 

as the source in the direct influence on A, and factor Bi is extensionally excessive in this 

influence. 

The proposed expert criterion of extensional proportionality, K S (B, A) is applicable to any 

pair of factors of a cognitive map, connected by direct influence. It allows to estimate 

whether factor-source of influence B is extensionally proportional to influence (or set of 

influences) on the receiver being modeled with link (B, A). 

For example, factor 2 in fig. 2 is extensionally excessive in the influence (2,1), according 

to K s (2,l) . The criterion of extensional proportionality for the influence receiver K D (A,B) is 

formulated and applied similarly, though in case of many influences onto one factor it is less 

informative at risk detection and error correction. 

Summary. The problem of risks due to the human factor in the field of formal methods of 

searching and making decisions in the control of complex and ill-structured situations is 

considered as the general problem for diverse models of subject-matter experts' knowledge 

and related formal methods (Abramova & Novikov, 2006; Abramova, 2007a). 

Earlier the idea about productivity of the uniform approach to the problem for diverse 

models of experts 7 knowledge, solved problems and formal methods has been stated, and 

some theoretical and empirical evidences in favor of this idea have been found (Abramova, 

2007a). 

The idea has found the reinforcement and further development at current studying risks 

concerned to causal influence transitivity in cognitive modeling, with carrying out analogies 

to risks in decision-making based on pairwise preferences. It is enough to say that just the 

analogy of principles of transitivity of paired preferences and causal influences has led to a 

hypothesis on possible violation of the axiom of causal influence transitivity, i.e. to risk of 

false transitivity at formal cognitive mapping what has been confirmed in practice. 

However, in few practical situations of violating the axiom of causal influence transitivity 

which we have analyzed by now, violations took place in cases when detection of false 

transitivity was perceived by experts as a signal of an error with necessity of cognitive map 

correction, and herewith correction was successful. Nevertheless, by analogy situations are 

conceivable when modeling of expert knowledge in terms of the chosen general formal 

model of cognitive maps appears impossible without separate violations of the axiom of 

influence transitivity, and the violations are estimated by subject-matter experts not as a 

signal of an error but as limitation of the formal model. 

The question on existence or possibility of such situations in practical cognitive maps of the 

complex and ill-structured situations is opened by today. Further research on the empirical 
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material of practical cognitive maps of complicated situations is planned with the aim to 
revealing such situations, their practical importance, and also rational experts 7 reasoning in 
such situations. 

3.2 SWOT-analysis on the basis of analysis of the model structure 

Generally, SWOT-analysis is expert determination of strength and weaknesses of SEO, 

opportunities and threats of its environment, and estimation of their interaction. Results of 

SWOT-analysis are represented as a matrix called " Window of opportunities 77 . 

The mathematical procedure of generation of matrix " window of opportunities 77 on the 

basis of analysis of structural features of SEO development cognitive map has been 

worked out. Thus, there is no need in regular attraction of experts with all accompanying 

procedures. 

The essence of the procedure of SWOT-analysis is the following. 

Analyzing a situation of SEO development, it is possible to put forward various hypotheses 

about desirable dynamics of any factor of the model. So, the parameter " attitude to factor 

dynamics 77 (AFD) is brought in for each factor of the model. If dynamics of a factor is 

positive (negative) AFD is equal 1 (-1). If it is difficult to evaluate the factor dynamics its 

AFD is equal to 0. The set of AFD vector on some set of model factors reflects desirable 

change of a situation in SEO. 

Let's designate through R ext (X ext ), R int (X int ) vectors of AFD of factors of the external and 

internal environment, where X ext u X int = X; X ext (0), X int (0) - vectors of initial trends 

accordingly. 

While the situation evolves each factor is being influenced not only by " neighbouring 77 

factors, but also by more " distant 77 ones and these indirect influences are transferred 

through chains of the appropriate factors and graph arcs that connect them. Set of influences 

both direct, and indirect to which each factor in a situation is subject to, is described with the 

use of concept of transitive closure of a cognitive map of the situation (matrix Q (3)). 

When constructing a matrix "Window of opportunities 77 , opportunities and threats of the 

environment, strengths, and weaknesses of SEO are determined on the basis of observation 

of the dynamics of model factors and estimations of their integrated influence on desirable 

dynamics of factors R ext (X ext ) m R int (X int ). The significance of strengths and weaknesses of 

SEO is determined as well. 

Let us introduce the basic definitions. 

Definition 1. If the initial trend of the internal environment factor, Xi int (0), is negative, i.e. does 

not correspond to a desirable direction of change (AFD), the given factor is regarded as a 

weakness of functioning and development of SEO, otherwise (the trend is favourable) - as a 

strength of object. The weaknesses determine internal threatening trends to SEO 

development, and strengths - internal favourable trends. 

Using a terminology of SWOT-analysis, we shall designate X st - a subset of factors the 

strengths determine of SEO, X w - a subset of the factor-weaknesses of SEO, X st ^jX w czX ]ni . 

Definition 2. The initial factor trend from X influences positively on desirable dynamics of 

the factor from X if the following equality holds true 

sign(xi(0) qij ) = rj(xj), 

where qij - (i,j) element of a transitive closure matrix Q, which determines integrated 
influence of i factor on the j factor; qij=0 if Xi does not influence Xj. 
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If AFD of some factors are given equal to zero (ij(xj) = 0) such factors are excluded from the 
analysis (integrated influences on them of other factors are not taken into account). 
The following definitions follow from definition 2. 

Definition 3. The factor of an environment Xi ext is neutral for X int , if the initial trend of this 
factor does not influence (qij=0) the desirable dynamics of all factors of the internal 
environment of SEO, X^. 

Definition 4. The factor of environment Xi ext characterizes the opportunity for SEO 
development if the factor is not neutral and its initial trends does not negatively influence 
(through the appropriate integrated influences) the desirable dynamics of all factors of 
internal environment of SEO, R int (X int ). In other words the initial trend of factor xi ext 
promotes SEO development in a desirable direction. 

Definition 5. The trend of the environment factor Xi ext threatens the SEO development, if it 
negatively influences (through the appropriate integrated influences) desirable dynamics 
even of one factor of internal environment Xj int eX int . 

Using a terminology of SWOT-analysis, we shall designate X°p - a subset of factors- 
opportunities for SEO development, X th - a subset of factors-threats to SEO development, 
Xopu Xthc X ext . 

Definition 6. The internal environment factor, xi int ,is neutral for X ext , if the initial trend of the 
factor does not influence (qipO) the desirable dynamics of the environment factors X ext . 
Definition 7. The internal environment factor, Xi int , promotes strengthening of the 
opportunity of environment Xj°p if Xi int is not neutral and its initial trend favourably 
influences (through the appropriate integrated influence) the desirable dynamics of the 
factor Xj°p. Otherwise Xi int promotes decrease of the opportunity of environment. 
Definition 8. The internal environment factor Xi int promotes parrying of threats of 
environment Xj th if Xi int is not neutral and its initial trend favorably influences (through the 
appropriate integrated influence) the desirable dynamics of the factor Xj th . Otherwise Xi int 
promotes strengthening of threats of environment. 
On the basis of definitions 1-8 SWOT-analysis comes to the following stages: 

1. Building of cognitive map of SEO development with extraction of external X ext and 
internal blocks of factors. Vector of initial trends of factors X ext (0) and X int (0) is set. 

2. AFD for each factor R^X^t), R^t(X^t) i s set. 

3. Strengths and weaknesses for each object (X st and X w ) are found on the basis of the 
vector X int (0). 

4. 4. Matrix of transitive closure Q (2) is used to build 

matrix " Window of opportunities ext-int" on the basis of which opportunities X°P and 
threats X th of environment and their importance (how great is their influence on factors 
of internal environment) are determined, X°PuX th czX ext ; 

matrix "Window of opportunities int-ext" with the purpose of determination of internal 
opportunities of SEO that can neutralize the threats of environment X th , and also the 
problems connected with possible negative influence of SEO on environment X ext ; 
matrix "Window of opportunities op-th". The analysis of interferences between 
opportunities and threats allows to reveal opportunities which promote parrying of 
threats; 

matrix "Window of opportunities st-w" for revealing the latent internal opportunities 
allowing to remove weaknesses of SEO due to advantages. 
As a result of the analysis all factors are being grouped into the following classes: S 
(Strengths), W (Weaknesses), O (Opportunities), T (Threats). Factors inside of each class are 
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being ranked according to the force of their favourable (unfavourable) influence on the 
factors of another class. This procedure lets us estimate the importance of strengths and 
weaknesses, opportunities and threats for SEO development. 

3.3 Structure and goal analysis of SEO development 

When setting the goals of a SEO development a decision maker doesn't always manage to 

trace if the goals he has set are inconsistent, i.e. reaching of a goal will prevent from reaching 

of another one. Inconsistency of goals can also be influenced by the chosen ways of their 

reaching. 

Thus, it is very important to reveal the contradictions already at the stage of goal setting. 

The technique of the structure and goal analysis of SEO development (Avdeeva et al., 2003) 

allows to determine integrated (direct and all possible indirect) influences of one factor on 

the other and due to it to reveal inconsistencies between goal and control factors. The 

structure and goal analysis also allows to determine the most effective controls. 

The goal of a situation development is described by a subset of goal factors of cognitive 

model. That means that the vector of goals of a situation development is a vector of values 

of goal factors (fixed goal), or a vector of directions of change of these values (unfixed goal). 

Definition 9. The fixed vector goal includes vector of goals Y* and preset values of trends of 

change of each goal Y* . 

The fixed vector goal is a point in m-dimensional space of trends of goal change. In other 

words, goal is a vector of some "ideal" values of trends of goal factors change. 

Definition 10. The unfixed vector goal includes vector of goals Y* and directions of 

favourable trends of change of its coordinates according to their AFDs. 

Vector of favourable trends is a vector of interests of a decision maker (analyst). Restrictions 

are not imposed on the value of favourable change of goal factors (the more - the better). 

Thus, the structural analysis of cognitive model of a situation development under control 

consists of the following stages: 

Stage 1 - analysis of goals (coordinates of a vector of goals) on mutual consistency in order 

to answer the question " whether the vector of goals (fixed or unfixed) is inconsistent, i.e. 

whether the reaching of any of goals (coordinates in a vector of the given goals) will prevent 

from reaching of other goals?" 

Let Y = \\j 1 ...y\ be a set of goal factors and r(Y) - a vector of desirable AFDs. 

Definition 11. Vector of goals Y is consistent if 

r j r k = sign(q ik ) for any yi ,y k e Y . (5) 

where q ik - (i, k)-th element of matrix Q (3). 

If (5) is fulfilled for goal factors y { , y k they refer to as consistent, otherwise these factors are 

inconsistent. 

When the consistent vector of goals is formed the desirable integrated change of any of goal 

factors will not result in undesirable integrated change of other goal factors in a vector of 

goals. 

Stage 2 - check of a consistency of the set of control factors with the given vector of goals, i.e. 

whether the change of the value of any control factor (with the help of the appropriate 

control) will promote reaching of some goals in a vector of goals and at the same time 

prevent from reaching of other goals of a vector of goals. 
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Definition 12. Vector of control factors is consistent with a vector of goals Y, if for each 
coordinate of a vector of control actions U = (u r ..u p ) it is possible to determine such sign, 

that for a resulting sign vector sign (U) it will be fair: 

r s = sign(q st )sign(u t ) for any u t e U,y s e Y (6) 

When control factors are consistent with the vector of goals and (6) is fulfilled, any change of 
control factors according to a vector sign (U) will not cause the change of any coordinate of a 
vector of goals Y in undesirable direction. Let U*(0) be a vector of control actions the signs of 
which are selected according to (6), and U*(0) - vector U*(0) in which all coordinates are 

replaced with their absolute values. The concepts entered above allow to formulate the 
following statement. 

Statement. If the selected vector of goals Y is consistent and the set of control factors is 
coordinated with a vector of goals it is possible to choose such vector of control actions U for 
which it will be fair 

|U\(0)| < |U* 2 (0)| -> Y(U\(0)) < Y(U* 2 (0)) 

where YfLf^O)) is a vector of changes of goal factors caused by activation of vector of 

control actions U* i (0),i = (l...m) / i.e. property of "domination" by modules of control 
transfers into property of "domination" by results of their influence on goal factors. 
In other words, more "intensive" control (with large absolute values of coordinates) will 
cause more "intensive" changes of coordinates of the goal vector in desirable directions. 
The mentioned definitions are used in situation analysis and modelling. Thus, violation of 
conditions of consistency of the selected vector of goals can help the analyst to understand 
the interaction of goal factors and set his vector of goals "more correctly", conforming to the 
situation. Analysis of the vector of selected control factors on consistency with a vector of 
goals will allow to resign inconsistent control actions and, on the contrary, to more actively 
use "advantageous" control factors, the change of which according to control actions 
affecting them will result in great favourable changes of goal factors. 

Stage 3 - estimation of efficiency of influence of control factors on all coordinates of the 
vector of goals. Such estimation is useful when choosing the most effective control factors 
the changes of which with the help of the selected control actions will provide the 
purposeful development of a situation. 

Formally, the parameter of efficiency E(u k ) of the control factor u k (i.e. the maximal 
positive effect from the change of u k ) is determined as absolute value of the sum of 
coefficients of influence of the given control factor u k on the goal factors multiplied by 
AFDs of the goal factors, i.e. 



E(u k ) = 



l^qk 



where ri is the AFD of the goal factor yi, 

qki - (k,i)-th element of matrix Q. 

Really, the maximal positive effect Ay from realization of control u k on the factor Xk is 

estimated as 
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Ay = 



i=l 



where the sign of action gk coincides with the sign of the sum X r^ki / an d ^s value is equal 

i=l 

tol. 

On the basis of results of SWOT- and Structure and goal analysis we form the set of vectors 

of goals of SEO and vectors of controls that reflect possible strategy of secure SEO 

development. On the basis of modelling controlled development SEO general strategy of 

SEO purposeful development is worked out in view of trends in environment. 

The combination of the stages described above enables one to diagnose and define problems 

in SEO development and find ways of its solution and form well-founded goals and strategy 

of SEO development. 

3.4 Approach to deriving of strategy for problems solving 

In general, the control of socio-economic system can be represented as construction of 
strategy for the system development, defining the main goals and general directions for 
their reaching, and its implementation. 

Revealing the system development problems influencing negatively achievement of 
strategic control goals is one of the key stages of construction of strategy for socio-economic 
system control. 

This section presents the general scheme of method for forming solution strategy for ill- 
structured problems on the base of linear dynamic models on the base of cognitive map in 
regard to socio-economic systems (Fig.4). 
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Fig. 4. General scheme of method for forming strategy for ill-structured problems solution 
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The control problem consists in transfer of socio-economic system into one of the states 

corresponding to goal image. At that, the proposed approach allows determining the system 

state in both values of model factors and rates of factor changes. 

In the variant of approach described in this paper the state of socio-economic system is 

defined by rates of change of model factors. Correspondingly, the dynamics of change in 

simulated situation is analyzed on the basis of factor trends. 

The goal image of socio-economic system defines desirable directions of changing of the 

system state from the position of control subject. Formally, it is represented as 

C = pec R(XC)) (7) 

where X c is the subset of goal factors, X c czX (X is the set of model factors), R(X C ) is the 
vector of estimations of factor dynamics defining desirable directions for changing of goal 
factors: 



R(x<) 



+1, if acceleration of x c { changing rates 
is desirable, 
-1, otherwise. 



The strategy for socio-economic system development problems solution consists of w 
strategic steps that define the sequence of system state changing 

where S° is the initial state, S c is the goal state (corresponding to goal image (7)), S 1 -^>S i+1 is 

the strategic step, at which the problem is revealed, and, on the base of its analysis, from the 

set of model factors X, the subset of local goals (goal factors) and the subset of controls 

(control factors) are selected. At that, changing of control factors results in desirable 

changing of goal factors. 

Each strategic step S 1 -» S i+1 includes the following: 

Revealing the problem on the base of self -development simulation for initial state at i-th step 

(Maximo v, 2001; Maximov & Kornoushenko, 2001). As a result, the problem becomes 

defined more precisely in form of the subset P 1 of factors, which changes do not correspond 

to the goal image. 

Diagnostics of the problem via construction of " subgraph of causes" and structure and goal 

analysis (Avdeeva, 2006; Avdeeva et al., 2003, Maximov & Kornoushenko, 2001) with the 

purpose of extracting the subset of local non-contradictory factors Y 1 from P 1 and searching 

variants of control (subsets of control factors) U • facilitating change of Y 1 in desired direction. 

Non-contradiction of the goal factors means that the desired change of any goal factor from 
Y 1 does not result in undesirable change of other factors form Y 1 . 

Simulation of the controlled system development consisting the base for forming various 
scenarios of the controlled development for resolving the problem applying obtained 
variants of control U- and carrying out the comparative appraisal of scenarios with the 

purpose of selecting control being optimal for this strategic step (Avdeeva et al., 2007; 
Avdeeva, 2006; Avdeeva et al, 2003; Maximov, 2001). 

Redefinition of initial condition for the next strategic step via transformation I^S 1 *) of the 
resulting state factor values S 1 * at current step (Avdeeva, 2006) taking into account results of 
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monitoring (control) of changes and mutual influences of factor values conditioned by the 

environment changes (Avdeeva et aL, 2007; Avdeeva, 2006; Avdeeva et aL, 2003). 

At the next step, the cycle is repeated since new initial conditions can result in new 

problems preventing the desired development of socio-economic system. 

The strategy construction process is finished with reaching satisfactory result that consists in 

reaching nearest approach to the desired system state corresponding to the goal image in 

conditions of given constraints (invariability of model structure, limitations on control 

resources, etc.) (Avdeeva, 2006). This means that, starting from some strategic step, 

estimation of purposeful development of the system given as functional of goal achievement 

degree does not change. 

4. Conclusions 

Experience of applying of various models and methods on the basis of cognitive approach 
(in Russia and abroad), as well as increasing interest of practical specialists to developments 
in this directions show expediency of development of this approach in control. At that, we 
should note some unsolved (or partially solved) problems. The problem of risks due to the 
human factor in the field of formal methods of searching and making decisions in the 
control of complex and ill-structured situations is considered as the general problem for 
diverse models of subject-matter experts' knowledge and related formal methods 
(Abramova & Novikov, 2006; Abramova, 2007a). 

Let us outline some directions of research planned by the authors within the framework of 
development of cognitive approach. 

• Development of theoretical principles, methods, and technologies for constructing 
models on the base of cognitive approach while studying ill-structured systems and 
situations. This direction supposes forming the main principles and system of criteria 
directed toward the following: 

increasing coordination and mutual understanding between participants of the process 

of resolving complex problem situation. 

increasing formalization authenticity for initial knowledge (representation) of the 

problem situation. 
The general conceptual scheme for control of model construction process for ill-structured 
system (situation) is developed. The criteria of appropriate transition from initial 
representation of ill-structured system (situation) in form of the cognitive map to one or 
another mathematical model defining further formal processing of initial representations are 
formed. Applicability of formal model is estimated reasoning from features and specificity 
of considered system (situation). 

The approach to formalization of initial representations of ill-structured problem in form of 
collective cognitive map with the purpose of generalization and agreement of different 
representations between problem bearers, which are competent in various object areas of 
knowledge. Solution of this problem rests upon the developed methods of conceptual 
structuring (Avdeeva, 2006; Avdeeva et aL, 2003), as well as criteria and particular 
technologies of forming and agreement of collective concepts (Abramova et aL, 1999). 
The established problem of risks caused by human factor is now under research directed 
onto the following: 

• generalizing results of empirical verification of hypotheses about control subjects put 
into typical methods of decision making support; 
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• making recommendations on estimation of adequacy of subject-formal methods for 
solving problems of control in ill-structured situations with describing the methods in a 
language using principles proposed at the first stage with selective empiric check; 

• developing conception of cognitive simulator for users of simulation systems on the 
base of cognitive maps taking into account cognitive features and typical risks caused 
by human factor in the life circle of the considered subject-formal methods; 

• making recommendations on revealing and blocking risks while applying the methods 
on the base of cognitive maps while solving practical control problems with selective 
empiric check of recommendations. 

• Development of instrumental tools for support of intellectual activity of subject in 
control of development of ill-structured systems and situations. By now, the software 
analytical system (STRICE - Strategy Intelligent Creation Environment) implementing 
the function of model construction on the base of cognitive maps, scenario simulation 
and comparative estimation of scenarios has been developed (Avdeeva et al., 2007, 
Avdeeva, 2006, Avdeeva et al., 2003). The modular architecture of the developed system 
allows to build it up with other tools for solving various control problems, as well as 
interaction with modern information and analytical systems (for example, systems of 
gathering and analysis of information, ERP-sy stems). 

Further development of this software system is aimed at the development of interface 
providing soft intellectual control of purposeful process of generation of formalized 
knowledge of subject (individual and collective). 
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1. Introduction 

During the last few years attention has become an important issue in machine vision. 
Studies of attentional mechanisms in biological vision have inspired many computational 
models (Tsotsos et al, 1995; Itti & Koch, 2000; Frintrop et al, 2005; Torralba et al, 2006; 
Navalpakkan & Itti, 2006). Most of them follow the assumption of limited capacity 
associated to the role of attention from psychological proposals (Broadbent, 1958; Laberge, 
1995). These theories hypothesize that the visual system has limited processing capacity and 
that attention acts as a filter selecting the information that should be processed. This 
assumption has been criticized by many authors who affirm that the human perceptual 
system processing capacity is enormous (Neumann et al., 1986; Airport, 1987). From this 
point of view, a stage selecting the information to be processed is not needed. Instead, they 
claim the role of attention from the perspective of selection for action (Airport, 1987). 
According to this new conception, the function of attention is to avoid behavioural 
disorganization by selecting the appropriate information to drive task execution. Such a 
notion of attention is very interesting in robotics, where the aim is to build autonomous 
robots that interact with complex environments, keeping multiple behavioural objectives. 
Attentional selection for action can guide robot behaviours by focusing on relevant visual 
targets while avoiding distracting elements. Moreover, it can be conceived as a coordination 
mechanism, since stimuli selection allows serializing the actions of, potentially, multiple 
active behaviours. 

To exploit these ideas, a visual attention system based on the selection for action theory has 
been developed. The system is a central component of a control architecture from which 
complex behaviours emerge according to different attention-action links. It has been 
designed and tested on a mobile robot endowed with a stereo vision head. Figure 1 shows 
the proposed control model. Sensory-motor abilities of the robot are divided into two 
groups that lead to two subsystems: the visual attention system, which includes the 
mechanisms that give rise to the selection of visual information, and the set of high-level 
behaviours that use visual information to accomplish their goals. Both subsystems are 
connected to the motor control system, which is in charge of effectively executing motor 
responses generated by the other two subsystems. 

Each high-level behaviour modulates the visual system in a specific way in order to get the 
necessary visual information. The incoming flow of information affects high-level 
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behaviours guiding their actions, so the attentional system also modulates the behavioural 
system. This double modulation results in two control loops running in parallel, affecting 
each other and working in cooperation to get a common objective. The control loop 
associated to the visual system controls ocular movements that lead to the fixation of a 
visual target. The other loop controls the sensory-motor activity of high-level behaviours. 
The attentional nature of the proposed system favours the simultaneous execution of 
multiple behaviours. In our model, action selection is solved by selecting a visual target. 
Fixation of a focus of attention ensures that only actions compatible with the selected 
information take place, solving this way the problem of coordination among behaviours. 
The sensory selection matter can be treated easily in comparison to the action selection 
problem. Actions could be contradictory, making more difficult the process of selecting the 
most suitable one. However, stimuli are selected according to the relevance of their 
properties in the current situation. At any given time and for any situation, the stimulus 
having the most suitable properties can always be found. It results in a sensory selection 
that provides the necessary information to carry out the most appropriate action for that 
situation. Extending this idea, attention acts as a means of action serializing, which is 
produced by the ordered execution of actions associated to the sequence of selected stimuli. 
The specific interleaving among actions comes from a time relation that links internal 
system requirements and external world features. 




Fig. 1. Attention-based control model 



2. The visual attention system 

The proposed attention system has been modelled as a set of components collaborating to 
select, fix and track visual targets according to different task requirements. Components are 
organized in different processing levels as shown in figure 2. The low-level components are 
related to image acquisition, motor control, as well as computation and maintenance of 
regions of interest (ROI's). Components of intermediate level extract sets of ROI features 
related to "what" (appearance information) and "how" (spatial information) issues. These 
features are used by high-level components, called target selectors (TS), to drive attention 
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according to certain top-down behavioural specifications. Attention control is not 
centralized, but distributed among several target selectors. Each of them drives attention 
from different top-down specifications to focus on different types of visual targets. At any 
given time, overt attention is driven by one TS, while the rest attend covertly to their 
corresponding targets. The frequency of overt control of attention of each TS is modulated 
by high-level behavioural units according to their information requirements. The fixation of 
a selected target is accomplished by two independent camera movements: a saccadic and 
tracking movement in one of the cameras and a vergence movement in the other. This 
allows controlling attention from monocular information while keeping stable binocular 
fixation. 

HIGH-LEVEL BEHAVIOURS 



Target selectors 

Selection c of attention 




' Properties Computation 

Aspect properties 
Cwhat") 



( ROI Maintenance 



n? 



ir position 



Visual processor 

images acquisition 
ROi detection 




Tracking Controller 

Keeping the focus of 
attention 



Vergence Controller 

Binocular fixation 



forccmrr. 



o 

73 
O 

o 

-z. 

H 
73 

O 

\— 

CO 

< 

CO 

H 

rn 



Fig. 1. Architecture of the proposed visual attention system 

The whole process constitutes a perception-action cycle that begins with image acquisition and 
detection of regions of interest and provides a motor response as a result of the selection of a 
focus of attention. The rest of this section details the different processing stages of this cycle. 
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2.1 ROI detection and maintenance 

In its first stage, system detects image regions that exhibit highly informative contents. To 
get stable results, the detection process must be invariant to different transformations, such 
as different scales or illumination changes. Several interest point detectors have been 
proposed in the literature (Lindeberg, 1998; Lowe, 1999; Mikolajczyk & Schmid, 2001). 
Among the existing approaches, the Harris-Laplace method, proposed by Mikolajczyk and 
Schmid (Mikolajczyk & Schmid, 2001), shows outstanding results regarding to important 
scale changes as well as different rotations, translations and changes of illumination. This 
method is highly suitable for our purpose. However, looking for a reduction in processing 
time, we propose a variant of the Harris-Laplace interest point detector. Our proposal 
consists in applying the Harris-Laplace method to the multiscale prism resulting from a 
centred section of the scale space representation (figure 3). This new representation exhibits 
a Cartesian topology that approaches the retinal structure of the human eye (Bandera & 
Scott, 1989). 




Fig. 3. Multiscale prism 

The multiscale prism keeps an eccentricity-dependent resolution structure. With this 
representation, foveal areas are represented at every resolution level, whereas peripheral 
areas only appear at the higher levels. As a consequence, Harris-Laplace method detects 
regions of any size at the fovea and big-size regions at the periphery (figure 4). This 
provides a significant reduction of processing time at the expense of some loss of visual 
information. Nevertheless, non-detected regions correspond to less informative areas of the 
periphery. They can be recovered at any moment placing the fovea onto these areas 
through camera movements. 

Regions detected by the above process are integrated in a map built on the camera reference 
frame. This map serves as a short-term memory that maintains information about recently 
perceived regions of interest. Each region is indexed by its angular position. This allows 
keeping information about regions situated outside the current visual field, so attention can 
quickly shift to previously seen regions. 

At every processing cycle, new detected regions are stored in this memory and the 
previously stored ones are updated according to the new visual information of the scene. 
The updating process consists in searching every stored region in the neighbourhood of its 
position inside the current scale space. The spatial search allows locating regions whose size 
changed due to, for example, a translation movement of the robot. 
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Fig. 4. Harris-Laplace regions from multiscale pyramid (upper images) and prism (lower 
images) 

Each region maintains an attribute of permanence time that is increased or decreased 
depending on the success of its position updating. Regions whose permanence time is under 
a threshold are considered forgotten and are thus removed from the memory map. For 
remaining regions, other attributes, such as raw image window, 2D velocity and attention 
time, are updated. The group of attributes computed during this stage forms a first set of 
properties that are extended in later stages of the system. To get different pathways of 
processing that allow computing properties in a separate and independent way, regions 
keep an identification number. This identification value allows integrating subsets of 
separately computed properties that come from the same original region. 



2.2 Computation of high-level properties 

At this stage, processing flow is divided into two subsystems that compute high-level 
properties of each region of interest. These two subsystems are an analogous of the "what" 
and "how" systems proposed in neuroscience (Milner & Goodale, 1995). This division allows 
a specialization of functions, dedicating specific resources to each subsystem and sharing 
what is common from low-level processes. The "what" system computes properties related 
to the visual appearance of regions. The "how" system extracts spatial information of each 
visual region, such as position, orientation and movement. 
The ''what" system 

Properties computed by the "what" system must provide region categorization from its 
visual appearance. To get consistent results in this categorization process, properties must 
be invariant to transformation of the point of view, changes of illumination, etc. Histogram- 
based descriptors, like SIFT (Lowe, 2004), RIFT and Spin (Lazebnik et al., 2005), provide 
stability against different image deformations. Experimental results obtained from several 
comparative studies (Mikolajczyk & Schmid, 2005) show better discriminative power of this 
kind of descriptors in relation to traditional descriptors, like banks of filters and differential 
invariants. Based on these results and considering a lower computational cost, we use RIFT 
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and Spin descriptors (Lazebnik et al., 2005) to represent aspect properties of regions. Both 
descriptors are invariant to scale and rotation changes and extract information about 
gradient orientations and image intensities, respectively. 
The "how" system 

The group of properties extracted by the second subsystem is composed of features of 
regions that describe their action possibilities. They must provide information about how to 
interact with each region. In our current implementation, properties computed by this 
component are: 3D position of the region computed using stereo matching; spatial motion 
from region tracking and 3D position; and planarity and plane orientation of the overall 
region gathered by homography estimation. 

2.3 Selection of the focus of attention 

The selection of the focus of attention is accomplished by multiple control components 
(target selectors) that individually determine the region to get the focus of attention 
following specific criteria. Distributed control of attention provides strong advantages 
compared to centralized control. Firstly, it provides a clearer and simpler design of the 
selection process. Secondly, it admits the coexistence of different types of visual targets, 
which is a key aspect of any task-driven attentional system. In this sense, we hypothesize 
that it is necessary to separate the way properties of regions are integrated for attentional 
selection according to behavioural objectives. For instance, in a generic navigation task such 
as going somewhere by following a predetermined set of landmarks, two types of visual 
targets can be distinguished: landmarks guiding navigation and potential obstacles. 
Integration of the different sets of properties characterizing both targets could turn any 
distracting area for the described task into a salient region, so defining global selection 
criteria including the two targets could be unfeasible. Moreover, although an effective 
integration of different kind of targets was achieved, the system can not guarantee to 
provide the necessary sequence of visual fixations that allow properly distributing attention 
time among them. If we put these ideas in the context of robot surveillance, a more 
important question emerges: how are reactive and deliberative abilities integrated? If 
attentional control is centralized, this issue becomes complicated. It is due to the fact that 
unexpected and foreseen things, such as moving and static objects, could be defined by 
contradictory properties. As a consequence, a centralized control strategy would cause 
either an excessive alerting state or a passive response to unsafe situations. 
Individual selection process 

Each target selector selects a focus of attention whose properties keep the greatest 
correspondence with its selection criteria. For this purpose, it computes a saliency map that 
represents the relevancy of each region according to its top-down specifications. This map 
acts as a control surface whose maxima match with candidate visual regions to get the focus 
of attention. 

There are several alternatives for computing the saliency map. The most representative 
models that integrates top-down and bottom-up influences in attentional control employ a 
weighting process where features are modulated by multiplicative factors representing top- 
down specifications (Frintrop et al., 2005; Navalpakkan & Itti, 2006). This integration 
method presents some limitations that can be observed in figure 5. This figure shows the 
behaviour of a weighting method for the selection of a visual target on the basis of two 
properties (Pi and P2) . For any value of Pi and ¥% saliency is computed as: 
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saliency = V\W\ + P2W2 



a) 



where w\ and w^ are top-down weights associated to Pi and P2, respectively. 
Assuming that both properties are equally relevant in the selection process, weights w\ and 
W2 should have the same value to express the desired target description. Hence, there is no 
distinction between considering simultaneous or separate matching with several properties, 
which limits the definition of potential selection criteria. Moreover, as it can be observed in 
figure 5, the resulting saliency distribution in the feature space (a) would cause some 
ambiguity in the selection process. Thus, as shown in (b), any horizontal line across the 
plane in (a) gathers stimuli with equal saliency even though they correspond to very 
different candidate regions. 





PI P2 

(a) Saliency distribution in the feature space of P^ and P 2 



(b) Different stimuli with the same saliency value 



Fig. 5. Saliency computation using a weighting method (see the text above for details) 

The proposed selection process overcomes the limitations described above by using fuzzy 
techniques. Fuzzy logic (Zadeh, 1965) provides a framework of development that is very 
suitable for our purpose, mainly for three reasons: 

• Properties used in the selection process present some degree of uncertainty and 
imprecision. Fuzzy logic allows dealing with this problem by treating each property as 
a linguistic variable. 

• Selection criteria can be easily defined using fuzzy rules that associate matching of 
properties to saliency levels. 

• Output of every rule can be fused together using a fuzzy inference process that 
integrates all selection criteria to provide a final saliency value. 

Following these ideas, the saliency map is obtained from a set of fuzzy rules that associate, 
for every visual region, the fulfilment of certain selection criteria (antecedents) to different 
saliency levels (consequents). Each premise of a rule antecedent is expressed by a linguistic 
label on a region property. These labels give rise to fuzzy sets defined in the domain of the 
properties that are relevant for the selection of the focus of attention. Thus, each selection 
criterion is expressed by a rule antecedent that determines matching between properties and 
associated fuzzy sets. The consequent part indicates the saliency value that is assigned to a 
region that fulfils that selection criterion. Since selection priority depends on the degree of 
fulfilment of antecedents and not directly on property values, we use a zero-order Takagi- 
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Sugeno model (Takagi & Sugeno, 1985). Hence, rule outputs are expressed through constant 
values that allow ordering regions according to the corresponding selection criteria. Thus, a 
rule of the proposed system is written as: 

IF Pi is Ai AND P 2 is Bj ... AND P n is X k THEN saliency =s ih . k 

where Pi, P2, ..., P n are properties of regions, A;, Bj, ..., Xk fuzzy sets associated to those 
properties and Sy ;...* a constant value expressing a saliency level. 

To assign specific values to each rule output, two types of rules are distinguished in our 
system: exclusion rules and selection rules. Strictly negatives values (considering a value 
as negative) are associated with exclusion rules, whose antecedents define regions that have 
no interest in the selection process and, therefore, should not be attended. Positive values 
are assigned to selection rules in such a way that rules that express greater fulfilment of top- 
down specification will take greater saliency values than those that define less important 
regions. 

To illustrate these ideas, a fuzzy system for an obstacle selector is shown below. In this 
example, robot must detect potential obstacles situated in the trajectory to a target position. 
Three properties are considered for determining the obstacle quality of any visual region: 

• Relative Depth (RD): it is defined as the relation between region depth and target depth. 
Both depth values are computed at a reference system situated in the ground position 
of the robot with its Z axis fixed in the straight trajectory to the target. This relation 
quantifies the proximity degree of a region to the robot, to the target or to both of them. 
A value greater than 1 of this property can be considered as a region situated outside a 
potential trajectory to the target. Three fuzzy sets are considered for this property 
(figure 6): NearR, NearT and Far. NearR considers depths that are nearer to the robot 
than to the target, NearT is related to regions situated in a nearer depth to the target 
than to the robot and Far is associated to regions situated behind the target. 

• Deviation (Dv): it quantifies the proximity degree of a region to the straight line between 
the robot and the target. It is measured by the distance between the central point of the 
region and that straight line. This property is partitioned in three fuzzy sets (figure 7): 
Low, Medium and High. 

• Height (He): Properties described above consider ground parallel distances between a 
region and the robot. However, those regions situated high enough from the ground 
should not be selected as obstacles, since they are not interfering areas in the way to the 
target. For this reason, height of regions is included as the third property that allows 
defining selection criteria of our system. This property allows deciding whether to 
include or not regions as potential obstacles, so only two fuzzy sets are considered 
(figure 8): Low and High. 

Once fuzzy partitions have been determined, the rule system is established by defining a 
fuzzy rule for each combination of fuzzy sets associated to the group of properties. For the 
current example, it results in a system with 18 rules, i.e., one rule for each combination of 
fuzzy sets of He, Dv and RD. Table 1 shows a simplified representation of the whole rule 
system. Each inner cell corresponds to a fuzzy rule whose antecedent is formed by the 
conjunction of three premises: the premise about Height (He) of the superior column, the 
premise about Deviation (Dv) of the inferior column and the premise about Relative Depth 
(RD) of the corresponding row. Inside each cell, the consequent value of the associated rule 
is shown. For instance, the first column of inner cells represents the following three rules: 
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IF He is Low AND Dv is Low AND RD is NearR THEN saliency =sm 
IF He is Low AND Dv is Low AND RD is NearT THEN saliency =sm 
IF He is Low AND D^ is Low AND RD is Far THEN saliency =Sii3 




Fig. 6. Fuzzy sets of the Relative Depth property 




Fig. 7. Fuzzy sets of the Deviation property 





0> 



Fig. 8. Fuzzy sets of the Height property 

Consequent values are set according to the excluding/ selecting criteria of each rule. In the 
above example, every rule containing a premise with the form "He is High" , "Dv is High" or 
"RD is Far" is considered an exclusion rule, since any region characterized by those 
properties does not interfere in the trajectory to the target. This group of regions should not 
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be selected as obstacles, which is expressed by assigning a negative saliency value to 
consequents of the corresponding rules (consequents in red of table 1). Remaining rules are 
selection rules whose positive output values must impose a saliency order. The specification 
of these output values is carried out by ordering rules from the most to the least adequate in 
the selection process. The most important regions for obstacle selection are those situated 
nearest to the robot. This implies that the maximum saliency value has to be associated to 
regions fulfilling the hypothesis "He is Low AND Dv is Low AND RD is NearR" '. On the other 
hand, regions at near positions to the straight line between the robot and the target are more 
important than those situated far away from the approaching line. It means that regions that 
meet the premise "Dv is Low" are prioritized over those that do not fulfil the premise. All 
these considerations allow imposing the appropriate order for consequents of selection 
rules, leading to the following output relations: sm > Sm. > sni > S122. According to these 
relations, rule consequents can be established. Also, other considerations, such as the 
relative relevance of one rule in relation to others, can be taken into account in order to 
provide additional running conditions. 
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Table 1. Tabular representation of the fuzzy rule system of an obstacle selector 

Once the set of rules is defined, the system is operative to evaluate visual regions according 
to their properties. This evaluation results in a saliency map that stores the saliency value of 
each region. Through this map, system discards regions with negative saliency and orders 
remaining regions from the most to the least relevant. Final selection is achieved by 
choosing those regions whose saliency differ less than a certain percentage from the 
maximum saliency value. This makes possible to decide whether to obtain several candidate 
regions to get the focus of attention or to select only the most salient one. 
Inhibition of return 

When a target selector obtains several candidate regions where focus of attention should be 
directed to, a mechanism for distributing attention time among all of them must be 
included. The mechanism used is an inhibition of return (IR) process that is applied as the 
final selection step of each target selector. IR process allows displacing the focus of attention 
from currently attended region to the next most salient one throughout time. For this 
purpose, it maintains an inhibition map that represents the attentional fixation degree of 
each area of the visual field. According to both inhibition and saliency maps a winning 
region is finally selected to become the next focus of attention. 
Global selection process 

The simultaneous running of multiple target selectors requires including a global selector 
that decides which individually selected region gains the overt focus of attention at each 
moment. 
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Target selectors attend covertly to their selected regions. They request the global selector to 
take overt control of attention at a certain frequency that is modulated by high-level 
behavioural units. This frequency depends on the information requirements of the 
corresponding behaviour, so, at any moment, several target selectors could try to get the 
overt control of attention. To deal with this situation, global selector maintains a time stamp 
for each active target selector that indicates when to cede control to that individual selector. 
Every so often, the global selector analyses the time stamp of every target selector. The 
selector with the oldest mark is then chosen for driving the overt control of attention. If 
several selectors share the oldest time stamp, the one with the highest frequency acquires 
motor control. Frequencies of individual selectors can be interpreted as alerting levels that 
allow keeping a higher or lower attention degree on the corresponding target. In this sense, 
the described strategy gives priority to those selectors with the highest alerting level that 
require faster control responses. 

2.4 Binocular fixation of the focus of attention 

Overt control of attention is achieved by binocular fixation, centring the selected visual 
target in both images of the stereo pair. This process keeps attention focused on the target 
until the next one is selected. 

Our proposal for binocular control is to divide the stereo fixation into two independent 
camera movements: a saccadic and tracking movement in one of the cameras and an 
asymmetric vergence movement in the other one. This separation results in a master-slave 
configuration that simplifies the global control. Moreover, it allows fixating visual targets 
with unknown spatial properties, so binocular focusing on the target can be achieved 
although attention is guided by monocular information (Enright, 1998). 
Tracking control 

In our master-slave configuration, the master or dominant camera is in charge of fixating 
and tracking the focus of attention. For this purpose, a hierarchical correlation-based 
method is employed using a multiscale prism representation of the target. This 
representation keeps information about target and its near areas, allowing false matching 
regions to be discarded according to target neighbourhood. 

The proposed tracking method starts locating positions of the highest resolution level that 
exhibit high similarity to the target. If more than one position is found, a new comparison is 
done at the next resolution level. This comparison discards regions whose neighbouring 
areas do not match with proximal areas around target. If more than one similar region is 
found again, the comparison process is repeated at the next resolution level. Thus, the 
method proceeds by an ascendant search of the target position in the scale space until a 
winning region is obtained. 

Figure 8 shows two frames of a sequence of tracking obtained while robot approaches a 
target. In this scene, three similar regions appear in the visual field. The one situated at the 
right constitutes the visual target. Images show the application of the hierarchical tracking 
method to both situations. For each level, regions of high similarity to the target are 
highlighted in green. Similar regions obtained from previous level that are discarded at a 
given level are marked in red. As it can be observed in this figure, the method employs an 
ascendant searching strategy that is maintained until a unique position of high similarity is 
found. 
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Fig. 8. Focus of attention tracking control 

Vergence control 

The function of an asymmetric vergence movement is to centre the visual target situated at 
the fovea of the dominant camera in the image of the slave camera. Hence, vergence control 
can be treated as a problem of maximization of the correlation coefficient measured between 
the central window of the dominant camera and the epipolar homologous window in the 
other one. Now then, the right size of that central window is dependent on the properties of 
the visual world, which makes it necessary to compute a new parameter: the size of the 
correlation window. To solve this problem, we introduce a hierarchical control method that 
carries out a descendent search in the scale space. This method is the inverse process used in 
tracking control. The reason for using a new strategy in vergence control is that the 
unknown size of vergence area impedes a reliable ascendant location of vergence position. 
In contrast, a descendent search provides both extension and position of the vergence area. 
To deal with different potential extension of vergence area, the central image window of the 
dominant camera is represented at different resolution levels using a multiscale prism. 
Initial search is done at the lowest resolution level. It consists of locating the position inside 
the widest central image window that maximizes correlation with the corresponding image 
patch of the dominant camera. If a position of high similarity is found, the next level tries to 
find a more precise vergence position by searching in a neighbouring window of the 
previously obtained position. If no initial position can be found, the complete search is 
repeated at the lower level. This procedure is applied for each level until a final vergence 
position is obtained in the highest resolution level. 

Figure 9 shows the proposed vergence method working in two real situations. Left images 
in (a) and (b) show the multiscale pyramidal representation of the image in the dominant 
camera. Central prism is delimited by the red squares of each level. Right images in (a) and 
(b) depict the multiscale representation of the image in the vergence camera. The search 
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window within each level is represented by a red rectangle. As shown in this figure, 
computing vergence position (marked with an "X") at every resolution level allows selecting 
the search window of the subsequent level. If a given level produces no result, the process 
maintains the width of the search window in the subsequent one, adapting its value to the 
corresponding resolution. This behaviour can be observed in the second level of image (b). 
Once the whole process is completed, vergence position is obtained in the highest resolution 
level. 




Fig. 9. Vergence control in real situations 



3. Attention-based control 

Intelligent robot control is carried out through pre-established links between the set of high- 
level behaviours and the visual attention system. Each high-level behaviour starts up an 
attentional control mechanism that provides a coherent visual behaviour according to its 
task-goals. This control mechanism is a specific target selector that responds to high-level 
activation and modulation by supplying the appropriate flow of visual information. 
Links between attentional control components and high-level behaviours provide the 
necessary organization of the group of actions that solve a certain task. The attention system 
always guarantees the selection of a visual region that constitutes the unique visual input of 
every high-level behavioural unit. This sensory selection results in an effective execution of 
a subset of actions from the total of potential actions associated to active behaviours. The 
final sequence of actions, which is produced as result of the serialized attentional access to 
visual stimuli, forms a control dynamic that gives rise to the desired compound behaviour. 
The proposed visual system models several kinds of attention that provide different high- 
level control dynamics: 

• Bottom-up attention: selection of a focus of attention depends on properties of visual 
regions in the scene. If any region maintains properties that agree with selection criteria 
of active target selectors, the focus of attention shifts to that region. In this sense, the 
system provides a bottom-up control on non-searched stimuli, allowing the robot to 
react appropriately to unexpected events. 

• Top-down attention: attention system is modulated from high-level processes through 
the activation of some target selectors that provide the appropriate visual behaviour for 
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the current situation. From this point of view, the acting context has a remarkable 
influence on the selection of a focus of attention, allowing global control to centre on 
task-relevant visual stimuli of the environment. 

• Overt attention: visual attention ensures the selection of a focus of attention that acts as 
the unique source of visual information for high-level control components. This 
selection process allows high-level behaviours to produce appropriate reactions to the 
attended stimulus, avoiding any conflict among behaviours. 

• Covert attention: coexistence of multiple target selectors allows keeping a "mental" 
focusing on several visual stimuli. It implies that it is possible to maintain an alerting 
state that allows displacing attention at any moment to any visual target for achieving 
different behavioural goals. 

These control aspects can be observed in the following example of attention-based control 
for solving a task of navigation following a set of landmarks. The system is formed by the 
components and connections depicted in figure 10. Each behavioural component defines an 
attentional dynamic by activating and modulating a specific target selector. Three high-level 
behaviours take part in this system: LANDMARK-GUIDED NAVIGATION, GO TO POINT 
and EXPLORE. They modulate three attentional selectors of landmarks, obstacles and 
unattended regions, respectively. 

Two situations are considered in this system. The first one occurs when the location of the 
landmark that the robot must approach is known. In this case, active behaviours are 
LANDMARK-GUIDED NAVIGATION and GO TO POINT. The first one activates a 
landmark selector and sends the aspect information of the landmark to be located. 
Attentional frequency on landmark is modulated by the NAVIGATION behaviour with an 
inversely proportional value to distance between landmark and robot. Thus, the less the 
distance to target position is, the greater the degree of attention on landmark is dedicated. 
When the NAVIGATION behaviour receives visual information that match up with the 
expected landmark, it sends to the GO TO POINT behaviour the target position in order to 
make the robot reach that position. To accomplish its goals, this last behaviour is connected 
to an obstacle selector, which allows shifting attention to regions situated close to the 
trajectory between the robot and the landmark. The GO TO POINT behaviour interprets 
incoming visual information as the nearest region that could interfere in the approach to 
landmark. Thus, it performs different actions according to the position of the attended 
region and the goal position. Firstly, if both positions are near enough, it is considered that 
the attended region is part of the target area and, consequently, it responds by approaching 
that region. However, if there is some distance between the target and the selected region, it 
is assumed that attention is focused on an obstacle that must be avoided. To get low reaction 
times in view of unsafe situations, obstacle selector is modulated with an activation 
frequency that is proportional to robot velocity. 

The second situation that is considered arises when either NAVIGATION behaviour gets no 
initial information about landmark or when, after locating it, no new information is received 
for a long time. To deal with these situations, NAVIGATION behaviour deactivates GO TO 
POINT behaviour and activates EXPLORATION. The function of this new behaviour is to 
get visual access to regions of the environment that have not been recently attended. This 
function is accomplished through its connection to a selector of unattended regions of the 



Attentional Selection for Action in Mobile Robots 



125 



visual system. This selector maintains a purely bottom-up dynamic that, together with the 
inhibition of return mechanism, leads to the visual exploration of the scene. Simultaneous 
execution of landmark and unattended region selectors give rise to the localization of the 
landmark that is being sought as soon as it appears in the visual field. As a consequence, 
landmark gains the focus of attention and system returns to the first described situation. 
Once the goal position is reached, NAVIGATION behaviour reinitiates the whole process by 
sending to its target selector the description of a new landmark. 
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Fig. 10. Control components of a landmark-guided navigation task 

The two situations described above are solved using a multi-target attentional control that 
exhibits several kinds of attention, providing different high-level control dynamics. On one 
hand, the system provides top-down attentional control on landmarks, allowing high-level 
processes to decide whether to approach a target position or to explore the environment to 
locate that position. When a landmark is located, the focus of attention alternates between 
landmark and potential obstacles, keeping overt control on one of the targets and covert 
control on the other one. This allows robot to react appropriately to obstacles while 
landmark position can be quickly recovered. In the exploring situation, searching is carried 
out through a bottom-up control where no target specifications are considered, so 
exploration depends exclusively on external world features. As in the previous situation, 
simultaneous covert control in landmark selector during exploration allows an immediate 
focusing on target once it is visualized. 



4. Hardware and software architectures 

Our attention-based control model has been tested in a five degrees of freedom mobile robot 
(three d.o.f. for the head and two for the base) that is endowed with a stereo vision head. It 
is equipped with digital PID controlled servos and two ISight Firewire cameras (Figure 11). 
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This model of robot has been developed in our Laboratory and is widely used for 
prototyping and algorithm testing. 




Fig. 11. Robots use for testing our attention-based control approach 

To obtain a real-time modular implementation, we have designed a component-oriented 
software architecture that exhibits an efficient and flexible structure. Every component of 
the attentional system depicted in figure 2, as well as high-level behaviours and other 
support elements, has been implemented as independent software components following 
the principles of components-based software engineering (Szyperski, 1998; He et al., 2005). 
The Internet Communication Engine (Ice) middleware platform (ZeroC, 2007) has been used 
for connection among components. Thus, each component has been written as an 
independent C++ application that includes Ice objects and proxies to communicate with 
other components. The resulting architecture is characterized by a flexible structure, a high 
degree of reusability of its elements and real-time running capabilities (Bachiller et al., 2007). 



5. Experiments 

The attention-based control model has been tested through several real experiments using 
our testing mobile robot for solving autonomous navigation tasks. In this section, results of 
two experiments are presented. Several frames of the navigation sequences are shown. For 
every frame, two different views of the corresponding scene are presented. The first one is a 
view from the robot represented by the two images of the stereo pair. Through these images, 
the focus of attention that is selected at every moment can be appreciated. The second one is 
a general view obtained from a camera situated in the ceiling of the room. In this second 
group of images, the attended area is highlighted in red to facilitate the association between 
the two sets of images. 

The control system employed in these experiments is the same that was described in section 
3. Navigation is guided by a landmark or a sequence of landmarks that the robot must 
locate and approach while potential obstacles in the way are avoided. 
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5.1 Experiment 1: approaching a landmark with detection and avoidance of obstacles 

In this first experiment, the robot must approach a landmark (star) situated in a wall in front 
of it. Two boxes stand in the way to the target position, so the robot must avoid them by 
going through the free space between them. On the floor, behind these two obstacles, 
several planar objects have been placed. The robot has to determine that these objects do not 
interfere in the trajectory to the landmark, since it can drive over them. Planar ity and 
orientation properties of regions provide this distinction between obstacles and ground 
areas. Detection of ground areas is depicted in the sequence of images of the stereo pair by 
green regions. This process is only carried out for regions situated near to the focus of 
attention, so not all the planar regions placed on the floor are detected as ground regions. 
Figures 12-15 show the result of this experiment. Initially (a), the robot is attending the 
landmark, causing the activation of the GO TO POINT behaviour. Consequently, attentional 
control responds by fixating the focus of attention on the first obstacle (b), which allows 
behavioural control to produce suitable avoiding actions (c-e). Proximity to the second 
obstacle gives rise to a shift of attention towards the corresponding area (f). This attentional 
change causes variations in the steering of the robot allowing the avoidance of this second 
obstacle (f and g). Once the robot is appropriately oriented, attention shifts again to regions 
near the target and new approaching actions follow (h). After some time, landmark selector 
recovers attentional control producing the updating of the target position (i). This updating 
causes new changes of attention that directs to proximal areas between the robot and the 
landmark. Specifically, attention focuses on plane regions of the floor (j), which allows high- 
level processes to detect them as ground regions. This detection gives rise to the correct 
treatment of the new situation that results in advancing movements towards target position 
(j-1). After several instants, attention is again centred on the landmark (m) making the robot 
to approach it in a more precise way (n-o) and finally reach the goal position (p). 
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Fig. 12. Experiment of navigation avoiding obstacle (first part) 
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Fig. 13. Experiment of navigation avoiding obstacle (second part) 
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Fig. 15. Experiment of navigation avoiding obstacle (fourth part) 



5.2 Experiment 2: navigation following a sequence of landmarks 

The aim of this second experiment is to test the robot behaviour when navigation is guided 
by a sequence of landmarks. 

The scenario of this experiment consists of two landmarks that the robot must sequentially 
reach. First landmark is formed by three concentric squares of different colours. The second 
one is the same landmark of the previous experiment. Figures 16-18 show results obtained 
in this test. At the beginning of the experiment (a), attention is fixated on the first landmark. 
Absence of obstacles allows a continuous advance towards the target position (b-d). Once 
this position is reached (d), the NAVIGATION behaviour reprograms landmark selector to 
locate the second landmark. Since there is no visual region that fulfils appearance properties 
of this new landmark, an exploratory behaviour is activated. Attentional control associated 
to this exploratory activity provides the visual access of the second landmark (g), once other 
visual areas have been explored (e-f). Since that moment, first landmark is conceived as an 
obstacle that robot must avoid for reaching the new target position. Obstacle selector 
provides this interpretation of the situation by centring attention on regions associated to 
the first landmark (h). Once the robot is appropriately oriented to avoid collisioning with 
the obstacle (i-j), attentional control focuses on regions situated near to the target. As a 
consequence, behavioural control produces approaching actions (k-1), which are interrupted 
when landmark selector recovers attentional control (m). After updating the target position, 
new approaching actions follow (n-o), allowing the robot to eventually reach the desired 
position (p). 
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Fig. 16. Experiment of navigation following a sequence of landmarks (first part) 
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Fig. 17. Experiment of navigation following a sequence of landmarks (second part) 
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Fig. 18. Experiment of navigation following a sequence of landmarks (third part) 

6. Conclusions 

In this chapter, a novel computational model of visual attention based on the selection for 
action theory has been presented. In our system, attention is conceived as an intermediary 
between visual perception and action control, solving two fundamental behavioural 
questions: 
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• Where to look? Attention drives perceptual processes according to action requirements. It 
selects relevant visual information for action execution. 

• What to do? Attentional selection limits potential actions than can be performed in a 
given situation. In this sense, actions are modulated according to perceptual result of 
attentional control. 

These ideas give rise to what we have called attention-based control model. The 
proposed global system is a behavioural architecture that uses attention as a 
connection means between perception and action. In this model, behaviours are 
control units that link attention and action. They modulate the attention system 
according to their particular goals and generate actions consistent with the selected 
focus of attention. 

Links between attention and action become effective through several properties of the visual 
attention system: 

• Top-down specifications: attention is modulated from high-level behaviours according 
to their acting requirements. Hence, attentional selection is strongly influenced by 
action. 

• Distributed control: attentional control is distributed between multiple individual 
control units that keep several visual targets simultaneously. This control scheme 
allows attention to be modulated from multiple behaviours with different information 
requirements. 

• Overt and covert attention: the system provides overt and covert control of attention, 
allowing attentional selection to alternate among several visual targets according to 
external world features and internal robot requirements. 

All these properties make possible to define proper interacting dynamics between 
behavioural and attentional controls that result in a global compound behaviour. Moreover, 
they provide the necessary coordination among individual behaviours that leads to an 
effective autonomous control. 

The whole system has been designed as a component-based software architecture that 
provides a real-time modular implementation. Real experiments have been conducted in a 
mobile robot for solving navigation tasks. Results show good performance of the system in 
real situations, providing initial groundings for deeper explorations of links between 
attention and action in robot control. 
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1. Introduction 

This chapter focuses on the estimation of car dynamic variables for the improvement of 
vehicle safety, handling characteristics and comfort. More specifically, a new estimation 
process is proposed to estimate longitudinal/ lateral tire-road forces, velocity, sideslip angle 
and wheel cornering stiffness. One main contribution of the process is that it uses 
measurements from currently available standard sensors (yaw rate, longitudinal/ lateral 
accelerations, steering angle and angular wheel velocities). Another contribution is that the 
process provides robust sideslip angle estimations with respect to cornering stiffness 
changes (or tire-road friction variations). Finally, the estimation process is applied and 
compared to real experimental data, notably sideslip and wheel force measurements. 
The last few years have seen the emergence in cars of active security systems to reduce 
dangerous situations for drivers. Among these active security systems, Anti-lock Braking 
Systems (ABS) and Electronic Stability Programs (ESP) significantly reduce the number of 
road accidents. However, these systems may be improved if the dynamic potential of a car is 
well known. For example, information on tire-road friction means a better definition of 
potential trajectories, and therefore a better management of vehicle controls. Nowadays, 
certain fundamental data relating to vehicle-dynamics are not measurable in a standard car 
for both technical and economic reasons. As a consequence, dynamic variables such as tire 
forces and sideslip angle must be observed or estimated. 

Vehicle-dynamic estimation has been widely discussed in the literature, e.g. (Kiencke & 
Nielsen, 2000), (Ungoren et al, 2004), (Lechner, 2002), (Stephant et al, 2006), (Baffet et al, 
2006a). The vehicle-road system is usually modeled by combining a vehicle model with a 
tire-force model in one block. One particularity of this study is that it separates the 
estimation modeling into two blocks (shown in Figure 1), where the first block concerns the 
car body dynamic while the second is devoted to the tire-road interface dynamic. 
The first block contains an Extended Kalman Filter (denoted as 01,4m) constructed with a 
four-wheel vehicle model and a random walk force model. The first observer 01,4x0 
estimates longitudinal/ lateral tire forces, velocity and yaw rate, which are inputs to the 
observer in the second block (denoted as 02,LAM). This second observer is developed from 
a sideslip angle model and a linear adaptive force model. 
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Measurements: yaw rate, steering angle, lateral acceleration 
longitudinal acceleration, angular wheel velocities 

i 



Block 1 
Observer Oi,4w 



Extended Kalman Filter 
Four-wheel vehicle model 
Random walk force model 



Longitudinal and lateral tire forces, yaw rate 

i 



Block 2 Extended Kalman Filter 

Observer 02,lam : Side-slip angle model 

Linear adaptive force model 



f 

Estimations : Longitudinal and lateral forces, sideslip angle, 
yaw rate, cornering stiffness 

Figure 1. Estimation process. Observers 01,4m and 02,LAM 

Some studies have described observers which take road friction variations into account 
(Lakehal-ayat et al, 2006), (Rabhi et al, 2005), (Ray, 1997). In the works of (Lakehal-ayat et al, 
2006) road friction is considered as a disturbance. Alternatively, as in (Rabhi et al, 2005), the 
tire-force parameters are identified with an observer, while in (Ray, 1997) tire forces are 
modeled with an random walk model. In this study a linear adaptive tire force model is 
proposed (in block 2) with an eye to studying road friction variations. 

The rest of the paper is organized as follows. The second section describes the vehicle model 
and the observer 01,4m (block 1). The third section presents the sideslip angle and cornering 
stiffness observer (02,LAM in block 2). In the fourth section an observability analysis is 
performed. The fifth section provides experimental results: the two observers are evaluated 
with respect to sideslip angle and tire-force measurements. Finally, concluding remarks are 
given in section 6. 



2. Block 1 : observer for tire-road force 

This section describes the first observer 01,4m constructed from a four-wheel vehicle model 
(see Figure 2), where ^is the yaw angle, /?the center of gravity sideslip angle, Vg the center 
of gravity velocity, and LI and L2 the distance from the vehicle center of gravity to the front 
and rear axles respectively. Vx,y,i,] are the longitudinal and lateral tire-road forces, 51,2 are 
the front left and right steering angles respectively, and E is the vehicle track (lateral 
distance from wheel to wheel). 

In order to develop an observable system (notably in the case of null steering angles), rear 
longitudinal forces are neglected relative to the front longitudinal forces. The simplified 
equation for yaw acceleration (four-wheel vehicle model) can be formulated as the following 
dynamic relationship (01,4m model): 



¥ = - 



L x [F yU cos(^) + F yU cos(£ 2 ) + F xll sin(^) + F xl2 sin(S 2 )] 

~ L 2i F y2l + F y22] 
E /^ F y\\ sin W ~ F y\2 Sin(£ 2 ) + F x\2 COS(£ 2 ) - F xll COS(S x )] 



(1) 
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where m the vehicle mass and Iz the yaw moment of inertia. The different force evolutions 
are modeled with a random walk model: 



FxH,F*i=[Ofil » = 1A y' = l,2 



(2) 




Figure 2. Four-wheel vehicle model 

The measurement vector Y and the measurement model are: 

Y = [v,r y ,r x ] = [YiJ2J3] 



Y x = ¥ 



-[F yU cos(^) + F yU cos(£ 2 ) + F y2l + F y22 + F xll sin(^) + F xl2 sin(£ 2 )] 



(3) 



7 3 = — [-F ' n sin(^) - F 12 sin(£ 2 ) + F xll cos(^) + F xl2 cos(^ 2 )] 



where yx and ^/ are the longitudinal and lateral accelerations respectively. 
The 01,41V system (association between equations (1), random walk force equation (2), and 
the measurement equations (3)) is not observable in the case where Fy21 and Fy22 are state 
vector components. For example, in equation (1, 2, 3) there is no relation allowing the rear 
lateral forces Fy21 and Fy22 to be differentiated in the sum (Fy21+Fy22): as a consequence 
only the sum (Fy2=Fy21+Fy22) is observable. Moreover, when driving in a straight line, yaw 
rate is small, SI and 52 are approximately null, and hence there is no significant knowledge 
in equation (1, 2, 3) differentiating Fyll and Fyl2 in the sum (Fyll+Fyl2), so only the sum 
(Fyl=Fyll+Fyl2) is observable. These observations lead us to develop the 01,41V system 
with a state vector composed of force sums: 
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X = ty,F yl ,F y2 ,F xl \ (4) 

where Fxl is the sum of front longitudinal forces (Fxl=Fxll+Fxl2). Tire forces and force 
sums are associated according to the dispersion of vertical forces: 



F, 



zl2 



-F xX 

F zll +F z\2 ""' J F z\l +F zl2 

F y\\ = — 7— F y\, F y\2 = ~ 7"= F y\ ( 5 ) 

^zll+^zl2 t z\\ +t z\2 

T7 ^z21 z? Z7 ^22 Z7 

^21 " ~ — *>2, ^22 " ~ — r yl 

t z2\ +t z22 t z2\+ t z22 

where Fzij are the vertical forces. These are calculated, neglecting roll and suspension 
movements, with the following load transfer model: 



= L 2 m S ~ Kog m 7x L 2 h cog m r_ 

zll ~ 2(L!+L 2 ) E{L X +L 2) 

_ L 2 m g- h cog m r x L 2 h cog m Yy 

zU ~ 2(L X +L 2 ) E(L X +L 2 ) 

= L \ m S + Kog m 7x L \ h cog m 7y 

z21 ~ 2(L l+ L 2 ) E(L l+ L 2 ) 

_ k m S + Kog m Yx L \hog m 7y 

zll ~ 2(L X +L 2 ) E(L X +L 2 ) 



) 

(6) 



hcog being the center of gravity height and g the gravitational constant. The superposition 
principle means that the load transfer model assumes the assumption of independent 
longitudinal and lateral acceleration contributions (Lechner, 2002). The input vectors LI of 
the 01,4:1V observer corresponds to: 

U = [S h S 2 ,F zlh F zX2 ,F z2h F z22 ] (7) 

As regards the vertical force inputs, these are calculated from lateral and longitudinal 
accelerations with the load transfer model. 

3. Block 2: observer for sideslip angle and cornering stiffness 

This section presents the observer 02,LAM constructed from a sideslip angle model and a 
tire-force model. The sideslip angle model is based on the single-track model (Segel, 1956), 
with neglected rear longitudinal force: 

■ _ F xX sin(S - P) + F yX cos(S - P) + F y2 cos(/?) • 
mV g 

Rear and front sideslip angles are calculated as: 
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= 8-/3-1, 



¥ 



Pl=-P + k 



¥ 



(9) 



where £ is the mean of front steering angles. 

The dynamic of the tire-road contact is usually formulated by modeling the tire-force as a 
function of the slip between tire and road (Pacejka & Bakker, 1991), (Kiencke & Nielsen, 
2000), (Canudas-De-Wit et al 2003). Figure 3 illustrates different lateral tire-force models 
(linear, linear adaptive and Burckhardt for various road surfaces (Kiencke & Nielsen, 2000). 
In this study lateral wheel slips are assumed to be equal to the wheel sideslip angles. 




Cl+ACal L^ar 

— Linear adaptive 



i — Burckhardt, asphalt dry 

- - - Burckhardt, asphalt wet : 
;- ■ Burckhardt, cobblestone wet 

— | - ■ ■ Burckhardt, ice 



2 4 6 8 10 12 14 

Front lateral slip, front wheel sideslip angle (°) 



16 



Figure 3. Lateral tire force models: linear, linear adaptive, Burckhardt for various road 
surfaces 

In current driving situations, lateral tire forces may be considered linear with respect to 
sideslip angle (linear model): 



F vi (P t ) = CtPt, i = 1,2, 



(10) 



where Ci is the wheel cornering stiffness, a parameter closely related to tire-road friction. 
When road friction changes or when the nonlinear tire domain is reached, "real" wheel 
cornering stiffness varies. In order a take the wheel cornering stiffness variations into 
account, we propose an adaptive tire-force model (known as the linear adaptive tire-force 
model, illustrated in Figure 3. This model is based on the linear model at which a 
readjustment variable ACai is added to correct wheel cornering stiffness errors: 



F yi (/3 i ) = (C i + AC ai )/3 i , 



= 1,2, 



(11) 
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The variable ACai is included in the state vector of the 02,LAM observer and it evolution 
equation is formulated according to a random walk model. Input U\ state X' and 
measurement Y' are chosen as: 

U'=[u 1 ',u 2 ',u 3 ',U4'] = [^,^,V g ,F xl ], 

X-[x 1 ',x 2 ',x 3 '] = [A AC al ,AC a2 ], (12) 

Y'=[y 1 ',y 2 ',y 3 , ] = [F yl ,F y2 , ry ]. 

The measurement model is 

yr=(C,+x 2 ')A 

y 2 '=(C 2 +x 3 ')/? 2 (13) 

y 3 '= -((Pi + x 2 ')A cos(u,') + (C 2 + x 3 ')/? 2 + u 4 'sin( Ml ')) 
m 

where 



pi=ui'-xi'-Li u y u ^ 

fr=- X i + L 2 % 



(14) 



Given the state estimation denoted as 

X'=[xi',x 2 \x 3 '] (15) 

the state evolution model of 02,LAM is: 

*l'= K sin ( w l ~ *l') + F yw\aux COS^'-Xj') + F 2 aux COSfo 1 ) - M 2 '] 

WM 3 

x 2 '=0 (16) 

x 3 '=0 
where the auxiliary variables Fywl,aux and Fyw2,aux are calculated as: 

F ywUux = (Q + x 2 ')(w 1 '-x 1 '-L 1 w ^/ ,) 

Fyw2,aux=(C 2 +X 3 'X-X 1 '+L 2 ^ 3 ,) 



(17) 



4. Estimation method 

The different observers (01,4m, 02,1AM) were developed according to an extended Kalman 
filter method. In 1960 R. E. Kalman published a paper describing a recursive solution to the 
discrete-data linear filtering problem (Kalman, 1960). Since this publication, Kalman' s 
method, usually known as the "Extended Kalman Filter", has been the object of extensive 
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search and numerous applications. For example, in (Mohinder & Angus, 1993), Mohinder 
and Angus present a broad overview of Kalman filtering. 

This paragraph describes an EKF algorithm. bs,k, be,k and bm,k represent measurement noise 
at time tk for the measurements, inputs and state evolution model respectively. This noise is 
assumed to be Gaussian, white and centered. Qs, Qe and Qm are the noise variance- 
covariance matrices for bs,k, be,k and bm,k, respectively. The discrete form of models is: 



F(X k ,U* k ) = X k+ \ tk+1 f(X k ,ul)dt 



X k+l=T( X k> U k) + b m,k (18) 

Y k =h(X k ,U* k ) + b s , k 
U* k =U k +b etk 

where / and h are the evolution and measurement functions. X k and X k are state 
prediction and estimation vectors, respectively, at time tk. The first step of the EKF is to 
linearize the evolution equation around the estimated state and input: 



A 8F 

* dX 



(xtvi) 



The second step is the prediction of the next state, from the previous state and measured input: 

X- k+x =F(xt,U* k ) (20) 

The covariance matrix of state estimation uncertainty is then: 

Pm = APk4 + B k Q e Bjt + Q m (21) 

The third step is to calculate the Kalman gain matrix from the linearization of the 
measurement matrix: 



Q -~rr:\X k+ i,U k J 



dX 
dU 



Ifik+uUt) (22) 



D k ^(x k+i ,ul) 
The following intermediate variables are used: 

R k = c k p k+\ c k + D kQe D k 

S k =B k Q e Di (23) 

T k = Pk+l^k + S k 
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and the Kalman gain matrix is: 



K k ~ T k( R k + Qs + c k s k +s k c k) 



(24) 



The estimation step is to correct the state vector in line with measurement errors: 

xi +l = X~ M + K k [r k+l - h(xi +l ,u* M )] (25) 

Finally, the co variance matrix of state estimation uncertainty becomes: 

p k+\ = p k+\ ~ K k( c k p k+i + s k ) ( 26 ) 



5. Observability 

From the two vehicle-road systems (01,4x0, 02,LAM), two observability functions were 
calculated. The two systems are nonlinear, so the observability definition is local and uses 
the Lie derivative (Nijmeijer & Van der Schaft 1990). 
The Lie derivative of hi function, at p+1 order, is defined as: 



L p f +l k= f l f(X,U) 

ex 



u f 



(27) 



With 



L\h i (X) = ^lf(X,U) 



(28) 



The observability function oi corresponding to the measurement function hi is defined as: 



dhi(X) 
dL l f h t (X) 



dL { J~ l) h t (X) 



(29) 



where n is the dimension of X vector and d is the operator: 

dhj = 



dh t dh t 
dxi dx 6 



The observability function of the system is calculated as: 

= (o h ...,o p Y 



(30) 



(31) 



where p is the dimension of the Y vector. Figure 4 illustrates observability analysis of the 
two systems for an experimental test, presented in section 6. Ranks of the two observability 
functions were 4 (for 01,41a) and 3 (for 02, LAM) (state dimensions) throughout the test, and 
consequently the state of the two systems were locally observable. 
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Figure 4. Ranks of the two observability functions for systems 01,4w and 02,LAM, during 
an experimental test (slalom) 



6. Experimental results 

The experimental vehicle (see Figure 5) is a Peugeot 307 equipped with a number of sensors 
including GPS, accelerometer, odometer, gyrometer, steering angle, correvit and 
dynamometric hubs. Among these sensors, the correvit (a non-contact optical sensor) gives 
measurements of rear sideslip angle and vehicle velocity, while the dynamometric hubs are 
wheel-force transducers. 




Figure 5. Laboratory's experimental vehicle 

This study uses an experimental test representative of both longitudinal and lateral dynamic 
behaviors. The vehicle trajectory and the acceleration diagram are shown in Figure 6. 
During the test, the vehicle first accelerated up to pc & 0.3 g, then negotiated a slalom at an 
approximate velocity of 12 m/s (-0.6g < yy < 0.6g), before finally decelerating to pc &0.7g. 
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Figure 6. Experimental test, vehicle positions, acceleration diagram 

The results are presented in two forms: figures of estimations/ measurements and tables of 
normalized errors. The normalized error sz for an estimation z is defined in (Stephant et al, 
2006) as 



S- 



100 T 



^measurement] 



max(z / 



measurement ) 



(32) 



6.1 Block 1: observer 01, 4w results 

Figure 7 and table 1 present 01,4w observer results. 



Estimates 


Max 


Mean 


Std 


¥ 


24.6°/ s 


0.4 % 


2.6 % 


Fyl 


5816 N 


3.1 % 


4.0 % 


Fy2 


3782 N 


2.9 % 


5.4 % 


Fxl 


9305 N 


3.1 % 


4.1 % 



Table 1. Maximum absolute values, 01,4w normalized mean errors and normalized 
standard deviation (Std) 

The state estimations were initialized using the maximum values for the measurements 
during the test (for instance, the estimation of the front lateral force Fyl was set to 5155 N). 
In spite of these false initializations the estimations converge quickly to the measured 
values, showing the good convergence properties of the observer. Moreover, the 01,4w 
observer produces satisfactory estimations close to measurements (normalized mean and 
standard deviations errors are less than 7 %). These good experimental results confirm that 
the observer approach may be appropriate for the estimation of tire-forces. 



6.2 Block 2: observer 02,LAM results 

During the test, {Fxl, Fyl, Fyl, Vg) inputs of 02,LAM were originally those from the 01,4w 
observer. In order to demonstrate the improvement provided by the observer using the 
linear adaptive force model (02,LAM, equation 11), another observer constructed with a linear 
fixed force model is used in comparison (denoted Orl, equation 10, described in (Baffet et al, 
2006b). The robustness of the two observers is tested with respect to tire-road friction 
variations by performing the tests with different cornering stiffness parameters {[CI, C2]*0.5, 
1, 1.5). The observers were evaluated for the same test presented in section 6. 
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1 ,4w, Front lateral tire-force Fy \ (kN) 




01,4w, Rear lateral tire-force F 9 (kN) 
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Ol, 4w, Front longitudinal tire-force F x ^ (kN) 
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Times (s) 
Figure 7. Experimental test. 01,4w results in comparison with measurements. Figure 8 
shows the estimation results of observer Orl for rear sideslip angle. Observer Orl gives good 
results when cornering stiffnesses are approximately known (fC_2, C_2]*l). However, this 
observer is not robust when cornering stiffnesses change ([C_2, C_2]*0.5, 2). 

Orl, Rear Sideslip angle (°) 




Figure 8. Observer Orl using a fixed linear force model, rear sideslip angle estimations with 
different cornering stiffness settings. 
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Figure 9 and table 2 show estimation results for the adaptive observer 02,LAM. The performance 
robustness of 02,LAM is very good, since sideslip angle is well estimated irrespective of 
cornering stiffness settings. This result is confirmed by the normalized mean errors (Table 2) 
which are approximately constant (about 7 %). The front and rear cornering stiffness estimations 
(Ci + ACi) converge quickly to the same values after the beginning of the slalom at 12 s. 



02,LAM 


0.5(C1,C2) 


(C1,C2) 


1.5(C1,C2) 


Max 


3.0° 


3.0° 


3.0° 


Mean 


7.4 % 


7.0 % 


7.2 % 



Table 2. Observer 02,LAM, rear sideslip angle estimation results, maximum absolute value, 
normalized mean errors. 
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Figure 9. 02,LAM adaptive observer, Sideslip angle estimation results, Front and rear 
cornering stiffness estimations Ci+ ACi, with different cornering stiffness settings 
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6. Conclusions 

This study deals with two vehicle-dynamic observers constructed for use in a two-block 

estimation process. Block 1 mainly estimates tire-forces (without an explicit tire-force 

model), while block 2 calculates sideslip angle and corrects cornering stiffnesses (with an 

adaptive tire-force model). 

The first observer 01,4m (block 1), an extended Kalman Filter, is constructed with a random 

walk force model. The experimental evaluations of 01,4x0 are satisfactory, showing excellent 

estimations close to the measurements and good convergence properties. 

The second observer 02,LAM (block 2), developed with an adaptive tire-force model, was 

evaluated for different cornering stiffness settings and was compared with an observer 

constructed with a fixed tire-force model (Orl). Results show that Orl is not robust when 

cornering stiffness parameters change, whereas 02,LAM gives excellent estimations of the 

sideslip angle. This result justifies the use of an adaptive tire-force model to take into 

account road friction changes. 

The different results show the potential of the two-block estimation process. The first block 

has the advantage of providing satisfactory force estimations without a tire-force model, 

whereas the second block provides robust sideslip angle estimations with respect to 

cornering stiffness changes (or tire-road friction variations). 

Future studies will improve vehicle-road models, notably for the calculation of the 

front/ rear sideslip angles, in order to widen validity domains for observers. Subsequent 

vehicle-road models will take into account roll, vertical dynamics and vehicle-tire elasto- 

kinematics. Moreover, experimental tests will be performed, notably on different road 

surfaces and in critical driving situations (strong understeering and oversteering). 
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1. Introduction 

This chapter presents a middle-term new load adjustment approach by overlapping for a 
set of jobs in a job-shop context, guaranteeing the existence of a limited capacity schedule 
without scheduling under the assumption of pre-emptive tasks. 

The proposed approach is based on the exploitation of the tasks scheduling time segments 
overlapping and on the distribution of the job's margins between tasks in a just in time 
context. First, we present a literature review concerning load adjustment approaches. 
Second, we introduce the overlapping load adjustment approach. Third, we present an 
original heuristic to use this approach in the case of job-shops organized firms. After that, 
we present the associated scheduling approach. Finally, we will discuss a more general use 
of this approach and the possible extensions. 

Generally, the production planning is made in a hierarchical way in two planning and 
scheduling decision levels. In the first step, we decide which products to supply, in which 
quantities and delays, in the second step, we adjust load to the capacity and schedule the 
tasks on the machines. 

There are three main and classical load adjustment approaches. First of all, we have the 
placement. This approach consists in calculating a detailed tasks schedule. A new task is 
integrated in the planning if we find a hole in the planning which is bigger than the duration 
of this task. This approach estimates only one schedule which can be destroyed by any 
disturbance. The second approach is the periodic and cumulative approach. It consists in 
calculating the cumulative load and capacity for a latest loading and for each period. This 
approach does not guarantee the existence of a scheduling solution because it does not take 
into consideration the ready dates constraints. The third approach is the periodic and non- 
cumulative approach. It consists in assigning tasks to periods and comparing period by 
period the available and the required capacities. This method estimates only the solutions in 
which the tasks are fixed in a specific period. 

The problem of sequencing decisions in production planning and scheduling was studied 
by (Dauzere-Peres & Laserre, 1999). They think that it is better in some cases to integrate the 
scheduling decision level in the lot sizing decision level and propose an iterative approach 
for planning and scheduling production. Some researchers integrated the scheduling and 
capacity constraints in their lot sizing model (see for example, Fleishmann & Meyr, 1997). 
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We can also find a survey on lot sizing and scheduling in (Drexl & Kimms, 1997). 
Nevertheless, a single machine is generally considered in most of these works which are 
difficultly applicable in a real situation. 

(Hillion & Proth, 1994) and (Gunther, 1987) studied the problem of finite capacity planning. 
(Hillion & Proth, 1994) considered the problem of a finite capacity flow control in a multi- 
stage/multi-product environment. (Gunther, 1987) developed two heuristics for the lot 
sizing in the context of finite capacity. (Neron et al., 2001) developed an approach for 
solving hybrid flow shop problem using energetic reasoning. This approach has some 
similarities in the concept with our approach. 

The problem of finite capacity planning in MRP based systems has also been studied. In 
order to obtain capacity-based production plans, (Pandey et al., 2000; Wuttipornpun & 
Yenradee, 2004) proposed a finite capacity material requirement planning system. 
(Grubbstrb'm & Huynh, 2006) considered multi-level, multi-stage capacity-constrained 
production-inventory systems in a MRP context. 

In comparison with all these approaches, the overlapping load adjustment approach allows 
to distinguish two main phases. The first phase consists in establishing a long or mid-term 
production planning where the feasibility is ensured without scheduling and tasks 
placement, which allows us to characterize a set of feasible scheduling solutions. The 
scheduling will be done only in the second phase. 

2. The overlapping load adjustment approach 

The time scale is divided into time periods. Each task of a job has got a processing time, 
requires one or more resources and has to be realized during a scheduling time segment 
associated with one or more consecutive periods. The scheduling time segments of consecutive 
tasks of the same job cannot overlap. From now on and throughout this paper a lapse of time 
called here lapse, designates a succession of a number of n consecutive periods. Let (a,b) be a 
lapse composed of a succession of periods which are limited by the periods a and b including 
them. The shortest lapses are composed of only one period, for instance (a,a). Such a lapse (a,a) 
is called a basic lapse. The longest lapse is noted (1,H) in which number 1 is associated with 
the first period of the planning time frame and the letter H the last one. From such a planning 
time frame, the total number of different lapses is equal to H*(H+l)/2.This number is of course 
to be multiplied by the number of existing processors. The sub-lapse of a lapse is a subset of 
one or more consecutive periods of this lapse. For instance, the sub-lapses of [1,3] are [1,1], 
[2,2], [3,3], [1,2] and [2,3]. Every lapse containing a lapse [a,b] is called the over-lapse of [a,b]. 
For instance, [1,3] is an over-lapse of [1,1]. Each lapse is characterized by: 

An accumulated capacity: sum of the capacities of each period included in this lapse. 

A direct capacity requirement: sum of the capacities required by the tasks whose 

scheduling time segment is exactly equal to this lapse. 

An accumulated capacity requirement: sum of the capacities required by the tasks 

whose scheduling time segment is fully included in this lapse. It is the sum of the direct 

capacity requirements of this lapse and its sub-lapses. 
(Dillenseger, 1993) sets the following proposition out: for any lapse, its accumulated 
capacity requirement must be equal or smaller than its accumulated capacity. He proves 
that it is a necessary and sufficient condition for the existence of a loading solution of the set 
of tasks (within the limits of their scheduling time segments and considering the capacity 
levels), according to pre-emptive possibility. 
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Let's consider the following example of a production plan composed of 7 jobs which will be 
treated on a single processor with ready and due dates as shown in Table 1 below. The 
considered period for this example is the week composed of five days (the day is the unit 
time): 



Job 


processing time 
(days) 


Ready date (beginning 
of the week) 


Due date (ending 
of the week) 


A 


3 


4 


4 


B 


1 


2 


4 


C 


2 


2 


4 


D 


2 


3 


3 


E 


2 


2 


2 


F 


1 


2 


3 


G 


4 


3 


4 


H 


2 


1 


3 


I 


2 


1 


4 


J 


1 


1 


1 



Table 1. Example 1 

Figure 1 represents a capacity requirement planning (CRP) corresponding to the example of 
Table 1. For instance, task F needs a load of 1 day and have a time scheduling segment 
composed of weeks 2 and 3 (it means that this task should be scheduled and produced in 
any time inside the weeks 2 and 3). The capacity of a period (the week in this example) is 5 
days. The margin of task F is so equal to 9 days (10 days of weeks 2 and 3 minus its load of 1 
day). The associated planning feasibility control graph (PFCG) is shown in Figure 1. For 
each lapse of weeks, we calculate the direct capacity requirement, the accumulated capacity 
requirement and the remaining capacity. For instance, the lapse [3,4] composed of weeks 3 
and 4 has a direct capacity requirement equal to 4 days (task G), an accumulated capacity 
requirement equal to 9 days (tasks with a scheduling time segment included in the lapse: A, 
D and G) and a remaining capacity equal to (10-9) days. The planning feasibility control 
graph proves the feasibility of this set of jobs (all the remaining capacities for all the lapses 
are positive). 

Firstly, this load adjustment approach was applied to plan the activities of a make-to-order 
company in a mono-level context (Dillenseger, 1993). This approach was applied then to a 
flow-shop composed of m processors (Bahroun, 2000a), to a generalised flow-shop 
(Bahroun, 2000b) and for the cyclic production context (Bahroun, 1999). 



3. Application for job-shop organized firms 

Let us consider N jobs with their due and ready dates. Each job is composed of one to m 
tasks realized on one to m processors with a certain order which is not necessarily the same 
for all the jobs (Figure 2). We suppose that these jobs are the results of a products supply 
calculation in an M.R.P. based system for instance. We aim at adjusting the load resulted 
by these jobs in a finite capacity way by adapting the overlapping load adjustment 
approach to the job shop case. 
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Fig. 1. Capacity requirements planning (CRP) and Planning feasibility control graph (PFCG) 
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Job 1: 
Job 2: 
Job 3: 
Job 4: 
Job 5: 



Processor 



1 I Processor 3 I Processor 4 I Processor 2 I 



Processor 



Processor 4 



1 I Processor 3 I Processor 4 I 
I Processor 2 I Processor 1 I 



Processor 2 



I Processor 1 



Processor 4 



) 



Fig. 2. Example of a job shop 

First, we will calculate the scheduling time segment of each task, considering the due and 
the ready dates of their job, their precedence constraint and the capacity constraints. After 
that, we will try to exploit the existing margins. They will be distributed on the different 
job's tasks and will be assigned with priority to the tasks corresponding to overloaded 
processors. We have developed a heuristic which tries to share out judiciously the job's 
margins on their tasks. For this, we calculate a latest loading on all the processors without 
margins (we only assign the margins of jobs with a unique task). We classify in the load 
decreasing order the processors. We then assign all the margins to the most loaded 
processor and after that we keep only the necessary margins to validate the processor 
loading (for any lapse, its accumulated capacity requirement must be equal or smaller than 
its accumulated capacity) and transfer the unused margins to the next processor accordingly 
to the load classification. Then, we reiterate the same treatment to the following processor 
until reaching the last processor. 
We define the following parameters: 

N = number of jobs. 

m = number of processors. 

Mai = the global margin of job i. 

pij = processing time of the task corresponding to job i on processor j. 

pij = if there is not a task of job i on processor j. 

ri = release or ready date of job i. 

di = due date of job i. 

bij = beginning of the scheduling time segment of the task corresponding to job i on 

processor j. 

dj = ending of the scheduling time segment of the task corresponding to job i on 

processor J. 

ACCP[a,b]j = accumulated capacity of the lapse [a,b] for processor j. 

ACCPR[a,b]j = accumulated capacity requirement of the lapse [a,b] for processor j. 

RC[a,b]j = remaining capacity of the lapse [a,b] on processor j. 

dp = duration of an elementary period. 
We note [x] the smallest integer which is greater than or equal to x and LxJ the biggest 
integer which is smaller than or equal to x. 

Our approach is based on five main steps (we will illustrate our approach with the example 
of Table 2): 
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Job 


Processing 
order 


Process. 

time 
on proc. 
1 (day) 


Process. 

time 
on proc. 
2 (days) 


Process. 

time 
on proc. 
3 (days) 


ready date 

(beginning of 

the period) 


due date 

(ending of 

the 

period) 


Global 

margins 

in periods 

(weeks) 


A 


P1->P2->P3 


3 


2 


2 


1 


6 


3 


B 


P1->P2->P3 


2 


3 


1 


1 


5 


2 


C 


PI 


3 








3 


5 


2 


D 


P2 





2 





2 


4 


2 


E 


P3 








2 


4 


6 


2 


F 


P3->P2->P1 


3 


3 


2 


2 


5 


1 


G 


P3->P2->P1 


1 


1 


1 


2 


6 


2 


H 


P3->P1->P2 


2 


2 


2 


3 


5 





I 


P3->P1->P2 


1 


2 


2 


3 


5 





J 


P2->P1 


3 


3 





2 


5 


2 


K 


P2->P3 





3 


4 


3 


6 


2 


L 


P3->P1 


2 





3 


4 


6 


1 



Table 2. Example 2 

1 st step : 

We calculate the global job's margins: 



Mai= di~ri~(JT |"(p ij / d p )l ) + 1 



(1) 



.7 = 1 



For our example, we consider a production system composed of three processors (PI, P2 
and P3) and a set of jobs (Table 1, d p = 5 days). We calculate the global margins using the 
last formula and we obtain the results reported in the last column of Table 2. After that, 
we determine the scheduling time segment of each task according to a latest loading 
without margins (we assign only the margins of jobs with a unique task like jobs C, D and 
E. In fact, these margins will be used only on a unique processor and will not be 
distributed on several processors). For instance, for job A, the last task on processor P3 
will have a scheduling segment that ends at the end of week 6 and will begin so, at the 
beginning of the same week (because the processing time of this task is inferior to a 
week), the task number 2 for the same job A on processor P2 will have a scheduling 
segment that begins and ends at week 5. The first task of Job A on processor PI will have 
a scheduling segment that ends and begins at week 4. Job C is composed of only one task, 
so we will assign its margin and the time scheduling segment will begin at week 3 and 
end at week 5. We calculate the scheduling segments of the other jobs in the same way 
and we obtain the capacity requirement planning of PI, P2 and P3 as shown in Figures 3 
and 4. 
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Fig. 3. Capacity requirements planning (CRP) of PI and P2 before treating 
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Fig. 4. Capacity requirements planning (CRP) of P3 before treating 

After that, we calculate the total load and the load by period for each processor as below: 



Processor 


Total load 
in days 


Number of concerned 
Periods (weeks) 


Load/ period 


PI 


20 


4 


5 


P2 


21 


4 


5,25 


P3 


19 


4 


4,75 



Table 3. Calculation of the processor load 

We classify and treat the processor in the decreasing order of the load/ period: P2, PI, P3. 

2 nd step: 

We assign all the global margins to the processor P2. Then, we calculate for each task the 

beginning and the ending periods of the scheduling time segment of this task: 

V i and for a processor j: 



e« remains the same 



bij = eji - Mai - [~(p ij / d p )l + 1 



(2) 



If we calculate the beginning time of the second processor of our example, we can generate 
the corresponding capacity requirement planning (CRP) and the planning feasibility control 
graph (PFGC): 



158 



Advances in Robotics, Automation and Control 



22 





5 



33 





5 



44 





5 



55 

4 


4 
1 



66 





5 



p eri 
3 



ods 



A2m 



B2(3) 



D2(2) 



J2£3j 



(wee 
4 



ks)X 
5 x 



F2(3) 



G2TTT 



K2(3T 



H2(2| 



12(2) , 



Accumulated capacity (days) 



10 15 



20 



25 



23 








10 




34 


3 


3 


7 




45 


4 





6 




56 


4 





6 



24 


11 


8 


4 




35 


11 


4 


4 




46 


4 





11 




36 


11 





9 



26 


21 





4 



Legend: 

Accumulated 
capacity requirement 



J 



Lapse [2,4] 




I 






24 


11 






8 


4 


/I 

Direct capacity Remaining 
requirement capacity 



Fig. 5. CRP and PFCG of the processor P2 after assigning all the margins 

3"* Step: 

We then check the feasibility condition. If we find problems in certain lapses (even with all 
available margins), we should lengthen the scheduling time segment of some tasks. If for a 
lapse [a, b] in the processor j, the validation condition is not verified: 

We consider in the duration increasing order all the tasks that bij > a et eij= b. 

In this list, we begin to treat the first task in the k th position in the list with a load equal 

or superior to the overloading in the lapse which allows us to delay the minimum 

number of tasks. 

We proceed progressively to the lengthening of the scheduling time segment of these 

tasks, period by period until the verification of the feasibility condition or arrival to the 

end of the list. 

If we arrive to the end of the list we try with tasks positioned in the (k-l) th , (k-2) th . . . 

position until the verification of the feasibility condition. 
In our example (Figure 5), we remark that we have an overloading in the lapse [2,5] which 
obliges us to delay of one period, one of the tasks included in this lapse and which finishes in 
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period 5 (A2, G2, H2, 12 or K2 as we can see in CRP of Figure 5). We choose in this example to 
lengthen the scheduling time segment of task G2 from the lapse [3,5] to the lapse [3,6]. 
Remark: If we do not accept to delay jobs, the lapses with negative remaining capacity 
indicate where we must increase the capacity by using for example overtime or interims. We 
can also introduce the notion of jobs priority for choosing which tasks must be delayed. 
4th step: 

Now, we will try to regain margins. We begin with the tasks corresponding to jobs with 
weak global margins. Tasks of jobs without margins are assigned to the elementary lapses 
([1,1], [2,2] etc.), those corresponding to jobs with one period margin are assigned to the 
lapses of the second column of the feasibility control graph, those corresponding to jobs 
with k periods margins are assigned to the column number k etc. Our treatment begins with 
the lapses of the second column because the corresponding jobs have only one period global 
margin and we must preserve these precious margins to validate the other processors and 
use the margins from jobs that have important global margins. 

A transfer of a task from the lapse [a,b] to the lapse [a+w,b], adds load to all the over-lapses 
of [a+w,b] which are not initially over-lapses of [a,b]. The transferred load must be equal or 
smaller than the remaining capacity on these lapses for maintaining the validation 
condition. 

The proposed approach for this transfer tries to transfer the maximum number of tasks and 
tries to match in the best way the transferred load in regard to the remaining capacity. 
Consequently, we construct the set of tasks which can be transferred, and we classify this set 
in the increasing order of their load. We transfer the tasks one by one in this order while the 
sum of their load is smaller than the remaining capacity. Then, we take the last task 
transferred and we try to change it by another task from the remaining tasks of the set and 
which matches better the remaining capacity. If two tasks have the same load we can choose 
for example the task corresponding to a product with a greater carrying cost. If we take the 
example of PFCG in Figure 5, we begin with the lapses of the second column. If we try, for 
instance, to regain margins from the tasks corresponding to the lapse [3,4], we should 
transfer the maximum number of tasks to the lapse [4,4]. We can transfer task F2 because the 
minimum of the remaining capacity of the over-lapses of [4,4] which are not over-lapses of 
[3,4] (the lapses [4,4], [4,5] and [4,6]) is 5 and it is greater than the load of the task F2. Then, 
we pass to lapses [4,5], [5,6], and next to the lapses of the third column (for the task G2, we 
succeed to regain 2 periods, the scheduling time is shortened from the lapse [3,6] to lapse 
[5,6]). We reiterate this treatment until arriving to the last column. We obtain after treatment 
of the processor P2 the capacity requirement planning of Figure 6. 
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Fig. 6. C.R.P. of processor P2 after treatment 
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We can generalize this step for a processor j: 

V the lapse [x,y] / x= a and y= a+w with 1 < w < H and 1 < a < H-w : 

Construct the set A = {set of tasks / b ij = a and e ij = a+ w} (The set of tasks that their 
scheduling time segment can be shortened, this set will be ordered in the increasing 
order of their load). 
Iterate then f or f = w -1 -^ 

> Cap = Min ( RC j [a+l+z, a+w+n] ) For n = 0^>H-a-w, z = f^>0 (we calculate the 
maximum load that can be transferred). 

> Transfer the maximum number of tasks from A (using the approach described in 
the precedent page) that the sum of their processing times is inferior or equal to 
Cap. Let C be the set of these tasks transferred and Q the sum of their processing 
time: 

V the task ij e C, we put b ij = a+1 + f 

For n = 0^>H-a-w and for z = f -^ 0, we do: 

RC j [a+l+z, a+w+n] = RC j [a+l+z, a+w +n ] - Q (We update the new time 

scheduling segment of the tasks and the remaining capacity of the concerned 

lapses). 
We update the beginning and the ending of the scheduling time segments of the other 
tasks on the other processors as follow: 

> If a task u precedes task j of the same job i on the processor j, we move its 
scheduling time segment in a manner that the ending time becomes equal to the 
beginning time of the task j. We effectuate the same treatment until arriving to the 
first task. 

> If a task u follows task j of the same job i on the processor j, we update in a 
symmetrical manner the scheduling time segment of this task and all the other 
tasks up to the last one. 

5 th step: 

We assign all the unused margins to the next processor (in this case the processor PI). We 
calculate the scheduling time segments of the tasks corresponding to this processor using 
the following formulae: 

V i, for a processor j : 

b^eji-M'ai-rfcij/d^l + l (3) 

Where M'aiis the remaining margin. 

We can assign margins for a task i on a processor j only if the precedent tasks of the same job 

are not already treated. We obtain the C.R.P. and the control feasibility graph of the 

processor PI of figure 7. 

We check then the feasibility condition. If for a lapse [a,b] in the processor j, the validation 

condition is not verified, we should lengthen the scheduling time segment of the tasks as 

below: 

We consider in the duration increasing order the list of all the tasks included in the 

lapse [a,b] and that bij = a or eij= b. 

In order to treat the minimum number of tasks, we begin to treat in this list the first task in 

the k th position in the list which has a load equal or superior to the overloading in the lapse. 

We try to regain a margin for this task by shortening the scheduling time segment of the 

precedent task of the same job in a processor already treated. 
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We try this for all the tasks positioned in the (k+l) th position in the list until succeeding 

or arriving to the end of the list. 

If we arrive to the end of the list, we try with tasks positioned in the (k-l) th , (k-2) th . . . 

position until succeeding or arriving to the beginning of the list. 

We must lengthen the scheduling time segment of as many tasks as necessary to 

validate the feasibility of the problematic lapse. 

If the remaining capacity continues to be negative, we reiterate the treatment of the 

tasks in the same order but by trying in this case to move completely if possible the 

precedent task of the same job in the past which allows us to lengthen the scheduling 

time segment of the tasks of this processor. 
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Fig. 7. CRP and PFCG of processor PI after assigning all the available margins 

If we do not succeed, we treat the tasks in the same order by trying to delay the due date on 
a minimum number of jobs or by increasing the capacity of the incriminate lapses. 



162 



Advances in Robotics, Automation and Control 



. Sim , 



aig\ 



Periods (weeks) ; > 

5 > 6 i 7 | 



CU3) 



\JJMZX 



\jim+ 



. FK3) , 



JH3) 



em 



8 ^1(2) , 



F3(2) 



Periods (weeks) 



, 63(11 
. E3 



, H3(2| 



^3iU 



, I3(2) , i 



-L 



A 3( 2) 



121 



L3(3) 



< K3(4\ 



Fig. 8. C.R.P. of processor PI and P3 after treatment 

If we take our example into consideration, the control feasibility graph of the processor PI 
indicates that we have a problem in the lapse [5,5] (Figure 7): the accumulated capacity 
requirement is 6 days and the available capacity is 5 days, so we should lengthen the 
scheduling time segment of one of the tasks Fl or Jl. If we lengthen the task in the right, the 
due date order will be delayed (Fl or Jl are the last task of the jobs F and J), so we will try to 
shorten or move the scheduling time of F2 or J2 in the processor P2. F2 can not be shortened 
(the scheduling time segment is one period). J2 can be shortened, the scheduling time 
segment will be shortened to the lapse [2,3]. We verify that the control feasibility graph of P2 
remains valid. The scheduling time segment of Jl will become [4,5]. The control feasibility 
graph of PI becomes valid and we can try to regain margins like done for the processor P2 
in the fourth step. We obtain the C.R.P. of processor 1 as shown in figure 8. 
Then, we apply step 5 for the last processor P3 and we obtain the C.R.P. as shown in Figure 
8. We update the beginning and the end of the scheduling time segments of the other tasks 
on the other processors as explained in step 4 and we obtain the final CRP of processors PI 
and P2 (figure 9). 



4. Scheduling 

In case where we admit to interrupt at least one task by period, the overlapping load 
adjustment approach furnishes a necessary and sufficient condition for the existence of a 
feasible scheduling solution. 
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Fig. 9. Final C.R.P. of processors P2 and PI after treatment 

The scheduling will be made in a real time manner. Indeed, in the end of each task the 
responsible will choose the next task between all the tasks that can be loaded and so on. 
This load adjustment approach will be coupled with a scheduling tool which can function as 
described below. If we take our example of Figure 1, the real time scheduling can be made 
as follow: 

In the beginning of week 1 we can choose to begin the task J, H or I (Figure 10), we 
choose for instance task J, then tasks I and H. 

We arrive at the beginning of day 1 of week 2 and we see that we can choose between 
three tasks: B, C, E or F (Figure 10). We choose for example tasks B and C and we obtain 
the partial scheduling of figure 10. 

We are in the beginning of day 4 of week 2 and we can choose between task E or F. The 
scheduling tool will inform the user if he can really choose one task and during how 
much time without breaking the feasibility condition. The user in this example can not 
choose task F because the task must be completed at the end of week 2 so the user is 
obliged in this case to choose E and after that F. 

We are at the beginning of day 2 of week 3, we can choose between task D and G. If the 
user chooses for example task G, the scheduling tool will inform him that the task G can 
be scheduled for only two days until the end of the third day of the week because there 
is the task D with a load of 2 days that must be scheduled on week 3. So, if the user 
decides to choose G, he must interrupt G after two days, schedule D and after that, 
continue with the task G, but the user can choose to schedule D and G without 
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interruption. We suppose that the user chooses task D and after that G. Finally, the user 
will schedule the last task A and we obtain the final scheduling of Figure 11. 
In our opinion, this original approach of scheduling presents many advantages in 
comparison with the automatic calculation of a schedule. First, the approach is really 
dynamic, each decision is taken in the last moment and we do not produce plans which will 
be out of date. Moreover, the user can have its own reasons to choose one task or another. A 
system that proposes and does not impose but exposes the consequences of each choice in 
regard to the scheduling which allows the user to decide with full knowledge of the facts 
and integrates his own criteria. This characteristic can promote the scheduling performance, 
and allow in all cases the responsibility of the user and his comprehension of the system. 




Fig. 10. the partial scheduling 




Fig. 11. the final scheduling 
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5. Experimentation 

An experimentation has been carried out on a set of examples. We will describe the 

conditions and the results of this experimentation. We have constructed 35 examples in a job 

shop context where we have varied the number of jobs (6 to 16), the number of processors (2 

to 4) and the size of the period (4 to 7 days). The total load is always near or superior to the 

capacity. We compared our approach with the classic approach of placement. For each 

example, we apply our approach and the placement approach. This placement is applied in 

two steps: first an earliest placement to determine the earliest due dates and second a latest 

placement using job's due dates as the maximum between the requested due dates and the 

earliest ones. 

The results of the experimentation are summarized in Table 4. We will use for this these 

notations: 

NMO: Number of Manufacturing Orders. 

NPr: Number of processors. 

PS: Period size in days. 

NPe: Number of periods. 

PTO: Processing time of the overlapping approach which represents the duration in days 

between the beginning of the first task of the M.O. until the ending of the last task of the last 

M.O. 

PTP: Processing time of the placement approach which represents the duration in days 

between the beginning of the first task of the M.O. until the ending of the last task of the last 

M.O for the placement approach. 

DPT: Difference in % for the processing time between the two approaches. 

ORO: Occupation rate of the processors in % for the overlapping approach. It is in fact an 

average of the occupation rate of all the processors. 

ORP: Occupation rate of the processors in % for the placement approach. 

DOR: Difference in % for the occupation rate between the two approaches. 

NDO: Number of delayed orders for the overlapping approach. 

NDP: Number of delayed orders for the placement approach. 

DDO: difference in the delayed orders between the two approaches. 

First, we have compared the two approaches in terms of processing times (figures 12 and 

13). We call the processing time the duration in days between the beginning of the first task 

on the first processor and the end of the last task on the last processor. The difference is 

about 4.5 % between the two approaches. 

Second, we have compared the two approaches in terms of occupation rate (figures 14 and 

15). We remark that there is a difference of about -4.3 % in the occupation rate for the two 

approaches. This fact is normal because with the first approach we try to keep the maximum 

of freedom in a middle term without giving a detailed production schedule. 

Third, we have compared the two approaches in terms of number of delayed jobs (figures 16 

and 17). The average number of delayed jobs with the overlapping approach is about 5.1 

whereas it is equal to 4.7 with the placement approach. We observe that the results given by 

the two approaches are very close even if the placement gives only one scheduling solution 

when the overlapping approach gives a set of scheduling solutions. 
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All these facts prove really the efficiency of our approach. However, we should experiment 
and compare our approach with other approaches and with a biggest number of examples. 
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Processing time for the two methods 




Fig. 12.The processing times obtained with the overlapping and placement approaches. 
Difference in %for the processing time between the two methods 




Fig. 13. The difference in % for the processing time between the two methods. 
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Fig. 14. The occupation rates obtained with the overlapping and placement approaches. 
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Difference in the occupation rate between the two methods 
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Fig. 15. The difference in the occupation rate between the two methods. 
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Fig. 16. The number of delayed orders for the overlapping and placement approaches. 

Difference in the number of delayed orders for the two methods 
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Fig. 17. The difference in the number of delayed orders for the two methods. 
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6. Conclusion 

The most common approach used in production planning remains MRP II (Manufacturing 
Resource Planning). The proposed approach in this paper constitutes an alternative to the 
traditional load adjustment approaches used in the CRP (Capacity Requirement Planning) 
modules in software based on MRP II philosophy. 

The new heuristic presented in this paper, in comparison with the usual middle and/ or 
long-term planning and scheduling approaches, has the following advantages: 

not setting a long-term tasks scheduling to assure that the planning can be properly 

carried out; 

exploiting and distributing judiciously the job's margins on their tasks and trying to 

respect the just-in-time principles; 

permitting the postponement of the final scheduling job's problems until the short term 

at the order release phase and/ or the scheduling phase; 

delaying, if necessary, the due dates of some jobs or increasing the capacity in some 

lapses for guaranteeing in every case the feasibility of the production planning. 
We can extend and improve our work by studying the possibility of introducing the 
overlapping of the scheduling time segments of consecutive tasks. We can also improve our 
heuristic in order to use it in the case where the tasks are not preemptive. 
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1. Introduction 

Providing human users with a structured insight into extensive data collections is a problem 
that, in the latter years, has gathered increasing attention by the scientific community. An 
even more daring and ambitious research challenge is the recent attempt to address the 
same problem within the scope of the so-called multi-agent systems. In this context, 
multiple autonomous and heterogeneous entities, i.e. the agents, populate the environment 
performing actions and taking decisions based on internal policies and their knowledge of 
the world. 

The courses of action of the agents and their interaction with the environment and with 
themselves generates extensive amounts of information. Underneath such flat data 
collections lies fundamental knowledge concerning strategies and common behaviors 
emerging from the agent actions. The advantage of extracting such "strategic" knowledge is 
twofold. On the one hand, it allows users a deeper insight into the multi-agent system, 
permitting to discover interesting patterns of actions that can be identified as interesting 
behaviors in the particular problem at hand. Consider, for instance, a multi-agent system 
simulating negotiation/ selling phases in a particular market (Viamonte et al., 2006): the 
detection of interesting patterns of actions can bring to light specific market strategies that 
can be exploited to optimize revenues. On the other hand, the identified strategic 
knowledge can be exploited by the agents themselves to improve their performance, for 
instance by updating their internal representation of the world and their behavioral policies. 
In order to extract such "strategic" knowledge, in this work we take a data mining approach 
using Association analysis, enriching it with the expressive power and the flexibility of 
ontologies. Association rules can effectively tackle both aspects of strategic behavior 
extraction, since they both provide a human understandable representation means for the 
extracted knowledge and an action rule base that can be used to supply agents with 
procedural and high-level knowledge concerning the identified strategies. 
Ontologies offer a structured description of the domain knowledge while maintaining data 
and their representation separated. An ontology refers to an "engineering artifact", 
consisting of a specific vocabulary containing the terms used to describe a certain domain, 
and a set of explicit assumptions regarding the meaning of vocabulary words. This set of 
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assumptions has usually the form of a logical theory, where terms correspond to unary or 
binary predicates, respectively called concepts and relations. For our purposes the logics 
supplies the formal language, the semantics and a set of proof procedures for representing 
the various aspects of the knowledge and for reasoning about it, in order to drive the data 
mining process. 

In general, data mining consists of a series of transformation steps, starting from data 
preprocessing up to post-processing, for converting raw data into useful information. The 
ontology drives the association rules extraction process intervening in the pre and post 
processing phases. Complex categories of domain-specific constraints are specified during 
the preprocessing phase, while during the post-processing phase, the extracted association 
rules are further filtered for providing better and more fitting results. 

Recent works have introduced the use of association rules for imitation learning in robotic 
agents within a reactive paradigm (Hellstrom, 2003), as well as in learning simple grasping 
and picking behaviors in agricultural robots (Zou et al., 2006), while (Makio et al., 2007) 
exploited association analysis to discover basic motor skills from human motion capture 
data. However, here we are not interested in determining low level motor behaviors in 
single agents, e.g. fight or fly reactions; rather, we seek to discover high level strategies 
underlying the actions of complex multi-agents systems. Within this respect, the 
introduction of ontologies for constraint-based multi-level association rule mining induces 
advantages in terms of the human interpretability of the strategies resulting from the 
Association analysis. Moreover, ontologies offer a simple mean for providing the agents 
with a representation of the extracted strategies that can be used, for instance, as high level 
primitives in the planning step of the agents' control hierarchy. 

The rest of the chapter is organized as follows: Section 2 introduces the data mining 
concepts that are used thorough the paper and describes related works in the area of 
behavior mining in agent-based systems. Section 3 formalizes the proposed model, while in 
Section 4 we describe the behavior mining approach and we present an example related to 
the simulated robotic soccer league. Section 5 concludes the paper with final considerations 
and future developments. 

2. Background 

The following subsections describe the background knowledge as well as the main concepts 
of Data Mining and Association analysis, together with a brief survey of the application of 
association rules to Behavior Mining in Agent-based Systems. Moreover we introduce the 
ontology theory that is used in the definition of the proposed model. 

2.1 Constrained association rule mining 

Knowledge Discovery in Databases (KDD) (Tan et al., 2005) is an emerging field that covers a 
wide range of applicative domains, several models for representing extracted patterns and 
models, and a large number of algorithms for data preprocessing, model extraction and 
model reasoning. KDD addresses the task of extracting " meaningful information'" from 
large volumes of data. Data Mining (DM) is an important part of the knowledge discovery 
process, whose aims is to convert raw data into useful information. DM consists of a series of 
transformation steps, from data pre-processing to post-processing. Pre-processing reduces and 
transforms raw input data into an appropriate format for the mining step. Post-processing 
ensures that only valid and useful results are retained. 
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Association analysis is a central task in DM. It aims at finding strongly correlated sets of 
events from large databases. In 1993, (Agrawal et al., 1993) introduced association rules for 
discovering regularities among products in large scale transaction data recorded by point- 
of-sale (POS) systems in supermarkets. For example, the rule {onions, vegetables} -> {beef} 
indicates that if a customer buys onions and vegetables together, he/she is likely to buy beef 
as well. Such information can be used as the basis for decisions about marketing activities, 
such as promotional pricing or product placements. In addition to the above example from 
market basket analysis, association rules are employed today in many application areas 
including web mining, intrusion detection, bioinformatics and so on. 

Nowadays, one of the most important open issue in association rules mining is due to the 
large number of extracted rules. In fact, it is not unusual for the collection of rules to be too 
large for being handled directly by an analyst. Moreover, often one would like to be able to 
focus the mining process on a particular portion of the search space or, in other words, it is 
important to have some additional expressiveness in order to define what is interesting and 
what is not. In order to solve these problems, frameworks for constrained association rule 
mining have been developed. 

In the remaining of this section, we formalize the problem of association rules mining, 
briefly introducing the Apriori algorithm; finally, we review the constraints-based 
association rule mining problem. 

Definition. Let I = {i\, . . . , i n ] be a set of literals, called items. Each item has some predefined 
attributes (e.g. price, type, etc.). Let D be a set of transactions, which form the database that is 
going to be disclosed. Each transaction t e D is an itemset, such that fcI.A unique identifier 
(TID) is associated with each transaction. A transaction t supports X, i. e. a set of items in I, if 
X c t. An itemset X has support s = supp(X) if s% of the transactions supports X. If | X | = k, 
then X is called a k-itemset. 

An association rule (AR) is an implication of the form X -> Y , where X <z!,Y czl and X n Y = 
0.Xvs> said antecedent (or body) and Y consequent (or head). An AR has a support s on D, if s% 
of the transactions in D contain X uY. An association rule has a confidence c on D if the c% of 
the transactions in D containing X, also include Y. In formulas: 

• supp(X ->Y) = supp(X uY) = \XuY\/\D\; 

• conf{X -> Y) = supp(X uY)/supp(X). 

It is worth noticing that, while support is a measure of the frequency of a rule, confidence is 

a measure of the strength of the relation between sets of items. 

An itemset is said frequent, if its support is higher than a user-specified minimum support. A 

rule is said strong if its support and its confidence are higher than a user-specified minimum 

support and a user-specified minimum confidence, respectively. 

A constraint on itemsets is a function C\2 l -> {true, false}. An itemset I satisfies a constraint C, 

if and only if Q (I) = true. Similarly, a rule constraint is a function Cr.R -> {true, false}, where 

R is the set of association rules. In general, given a conjunction of constraints, the 

constrained association rule mining problem requires to compute all the rules that satisfy 

the constraints. 

Algorithms for AR mining. Association rule mining algorithms scan the database of 

transactions and calculate the support and confidence of the candidate rules to determine 

whether they are significant or not. Their computational process is composed of two steps: 

(1) given a minimum support, find all frequent itemsets; (2) given a set of frequent itemsets 

and a minimum confidence, find all the association rules that are strong. Step (1) is, usually, 
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harder than rule generation and the main difficulty of itemsets generation comes from the 

large size of the search space. 

Apriori has been the first frequent itemset algorithm to be proposed (Agrawal et al., 1994). It 

exploits a bottom-up, level- wise exploration of the lattice of frequent itemsets. During a first 

scan of the dataset, all the frequent singletons, denoted with Li, are found. Next, Li is used 

to find the set of frequent 2-itemsets Li, and so on until all the frequent patterns have been 

discovered. 

Each iteration is composed of two steps: candidate generation and support counting. During the 

first step, a collection of possibly frequent itemsets Q of length k is generated, and their 

actual support is calculated with a single scan over the dataset during the second step. In 

principle, it would be easy to generate all the possible k-itemsets at each iteration, given that 

I is known. As a result, we will have an extremely large collection of itemsets that have to 

pass the support counting phase. The Apriori algorithm exploits a pruning of the search 

space in order to reduce the number of candidates before each iteration. Pruning is based on 

the concept that, if X eQ has a subset of length k - 1, that does not belong to Lm i.e. the 

subset is not frequent, X cannot be frequent as well, and it can be removed from Q. 

A survey of constraints-based AR mining. The fundamental idea inspiring constrained AR 

mining is to introduce a set of constraints that association rules should satisfy. The analyst 

can use a conjunction of constraints for specifying the properties of the patterns of interest. 

As a trivial example, let us focus on market basket analysis. Lets suppose that each item is 

associated to a fixed price, and that we are interested only on those co-occuring set of items 

such that their average price is higher than a given threshold. KDD systems should be able 

to exploit such constraints to speedup the knowledge extraction process. 

The most important classes of constraints and their properties are shown in Fig. 1. For a 

complete and formal definition please refer to (Bonchi & Lucchese, 2007). 

The first interesting class is the anti-monotone (Agrawal et al., 1993), which was already 

introduced with the Apriori algorithm (i.e. the minimum frequency is an anti-monotone 

constraint). It simply states that, if a predicate holds for a set S, then it holds for any subset 

of S. At the opposite, the monotone (Bucila et al., 2002) constraints state that, if a predicate 

holds for a set S, it holds also for any superset of S. For example, the constraint max(S.price) 

> 100 is monotone, while max(S. price) < 100 is anti-monotone. 

The succinct constraint (Han et. al., 1999) states that the decision concerning the satisfaction 

of a predicate can be determined based on the single element of the S set. For instance, the 

max constraints defined above are both succinct, since a single element of S can be used as a 

comparator. The avg(S) constraint is not succinct. 

The class of convertible anti-monotone (resp. convertible monotone) (Pei & Han, 2000) includes 

constraints Ccam (resp. Ccm) that define an item ordering relation such that whenever an 

itemset S satisfies Ccam (resp. violates Ccm), so does any prefix of S. For instance, 

mean(S. price) > 100 is a convertible anti-monotone constraint, with respect to the descending 

price order. In fact, Ccam(S) => Ccam(T), where S is a prefix of T. 

The class of loose anti-monotone constraints, introduced by (Bonchi & Lucchese, 2005), simply 

states that, given an itemset S with \S\ > 2, a constraint is loose anti-monotone (denoted 

Clam) if: Clam(S) => 3ie S: Clam (S \ {i}). It is trivial to show that variance(S. price) < 100 is a 

loose anti-monotone constraint. 
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Fig. 1. Characterization of the classes of constraints. 

For each of these classes, there exists a specialized algorithm, which is able to take 
advantage of its peculiar properties. Techniques for constraint-driven pattern discovery can 
also be classified, depending on the KDD phase in which they are evaluated, in the 
following categories: 

• pre-processing constraints are evaluated during the pre-processing phase, restricting the 
source data to the instances that can generate only patterns satisfying the constraint; 

• mining constraints are directly integrated into the mining algorithm (e.g. Apriori) used 
for extracting the rules; 

• post-processing constraints are evaluated for filtering out patterns after the mining 
algorithm. 

(Wojciechowski & Zakrzewicz, 2002) focuses on improving the efficiency of constraint-based 
frequent pattern mining, by using dataset filtering techniques that conceptually transform a 
given data mining task into an equivalent one operating on a smaller dataset. 
A first preliminary effort to propose a general framework for constrained frequent itemsets 
mining is (Bonchi & Lucchese, 2007), in which the authors present the design of 
ExAMiner GEN , a general Apriori-like algorithm, which is able to exploit all the possible kinds 
of constraints presented above. In particular, the authors suggest a data reduction technique 
for obtaining advantages from the conjunction of monotone and anti-monotone constraints. 
They observe that a transaction that does not satisfy the monotone constraint CM, can be 
removed since none of its subsets will satisfy CM either, and therefore the transaction 
cannot support any valid itemset. This data reduction, in turn, lowers the support of other 
frequent but invalid itemsets, thus reducing the search space and improving anti-monotone 
pruning techniques. This virtuous circle is encoded level-wise in the Apriori algorithm. 
Constraints can also improve the comprehension of the extracted rules, with the aid of a 
concept hierarchy. A concept hierarchy is a multilevel organization of the various entities or 
concepts defined in a particular domain. For example, in the market basket analysis a 
concept hierarchy has the form of an item taxonomy describing the "is-a" relationships 
among the items sold in a store, e.g. "milk" is a kind of "food" and "DVD" is a kind of "home 
electronics" equipment. Mining on concept hierarchies is also called constraint-based 
multilevel DM and association rules extracted with the aid of a taxonomy are also called 
multi-level. 

The main advantages of incorporating concept hierarchies into association analysis are: (i) 
items at lower levels may not have enough support to appear in any frequent itemsets and 
(ii) rules found at lower levels of a hierarchy tend to be too specific and may not be 
interesting as rules at higher levels. 
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Methods to define and integrate item constraints on a concept hierarchy are originally 
introduced in (Agrawal & Srikant, 1995) and (Fu & Han, 1995). Originally, multilevel 
constraints are evaluated during the pre-processing phase, generalizing items at bottom 
level to higher levels of the hierarchy before applying Apriori. Basically, each transaction t, 
is replaced with its " extended transaction", which contains all the items in t along with their 
corresponding ancestors. (Srikant et al., 1997) and (Han et al., 1999) can be seen as the first 
attempts to integrate both constraint-based and multilevel mining directly into the rule 
extraction algorithm, employing anti-monotone proprieties existing across levels: "if a high- 
level item is infrequent, none of its descendants can be frequent". 

In (Goethals & Van den Bussche, 2003) the authors integrate the evaluation of constraints 
into the mining process, so that the amount of computation is proportional to what the user 
gets, limiting the exponential generation of patterns and rules. The constraints considered in 
this work are Boolean combinations of atomic conditions, e.g. Body(i) (or Head(i)) specifies 
that a certain item i occurs in the body (respectively, in the head) of a rule. Another 
interesting work on integrating constraints evaluation in the mining process is represented 
by DualMiner (Bucila et al., 2003). This system implements a new algorithm for finding 
frequent itemsets, that efficiently prunes its search space by employing both monotone and 
anti-monotone constraints. 

Other remarkable constraint-based optimizations can be found in (Gallo et al., 2005) and 
(Esposito et al., 2006). The former proposes an "incremental" approach exploiting the 
results of previous queries in order to reduce the response time to new queries. In this 
context the term incremental describes the fact that the outcome of a new query starts from a 
previous result. The proposed algorithm is able to deal with context-dependent constraints. 
A similar idea is used in (Esposito et al., 2006), where a new query is rewritten in terms of 
union and intersection of the result sets of other previously executed and materialized 
queries. The authors propose a two-step approach. Firstly, they find conditions to apply 
various typologies of constraints (essentially on items properties), then, by using an 
optimizer, they recognize equivalent queries avoiding repeated heavy computations. 
Finally, (Wang et al., 2003) explores support constraints, that specify what minimum support 
is required for each itemsets, so that only the necessary itemsets are generated. The three 
constraints suppi(Bi, B3) > 0.2, suppi(B^) > 0.4 and suppffii) > 0.6 are examples of support 
constraints where each B\, called bin, is a set of disjoint items which, taken in isolation, do 
not need to satisfy any minimum support requirement. 

In this chapter, we focus on pre-processing and post-processing multi-level constraints. 
However, we plan to develop further our model by integrating constraints evaluation 
directly into the mining phase, taking advantage of the peculiar properties of each 
constraint. 

2.2 Behavior mining in agent-based systems 

In last twenty years, data mining has been extensively applied to provide users with a deep 
insight into data collections, supplying powerful tools for extracting high-level knowledge 
from extensive, often poorly structured, information sources. Recently (Cao et al., 2007) 
attention has started to shift from the classical view of data mining as a tool for transferring 
knowledge from raw data to human agents (i.e. the users), to a novel interpretation where 
data mining serves to transfer knowledge from data collections to intelligent artificial 
agents, under the form of strategic patters of actions, i.e. behaviors. 
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At the beginning, most of the works focused on learning single agent behaviors from 
demonstrational data: in (Hellstrom, 2003), for instance, a robotic system based on the 
reactive paradigm that learns complex behaviors by imitation is presented. Through a 
stimuli/ response mapping, the system determines the appropriate robot action as a direct 
function of its sensory inputs. This mapping is provided by a set of association rules that are 
extracted from sensory/ action data gathered as the human user exemplifies the behaviors 
that have to be learned by the robotic agent. Similarly, (Zou et al., 2006) use association rules 
to learn picking behaviors for agriculture robots. Other recent works focused on data mining 
as a tool for extracting motor primitives from motion data: (Makio et al., 2007) discusses 
how to discover motor skills by mining human motion capture data to extract association 
rules and frequent patterns that describe the dependencies among the body parts and the 
relative movements. In (Xu et al., 2007), on the other hand, association rules are used to 
discover and describe knowledge of chewing behavior in an object oriented framework. 
(Mori et al., 2005) collects information on the household behavior of people in daily life: by 
means of association rules, they discover frequent combinations of events, called episodes, 
that are used to predict and support the daily actions of the user. 

The models described so far relate to single agent scenarios (either robotic or purely 
software), but much of the recent works have been devoted to multi-agent systems. These 
ones differ from single agent systems in the sense that there is no global control and globally 
consistent knowledge and, since data and control are distributed, multi-agent models have 
the inherent advantages of distributed systems, such as scalability, fault-tolerance and 
parallelism. Notice that the term "multi-agent" refers to two aspects of the problem: the 
former and most straightforward one relates to the fact that the system itself is, indeed, 
composed of multiple agents. The latter aspect relates to the fact that the data used in the 
mining process is produced by multiple sources, i.e. by multiple human or artificial agents. 
This is the case of several agent based models with application to information retrieval, 
computational economics and robotics. In (Viamonte et al., 2006), for instance, is presented a 
Multi-Agent Market Simulator where the agents behavior follows the customers and seller 
profiles extracted by mining information from the market segment under analysis. (Bezek et 
al, 2006) presents an algorithm for extracting strategies from a RoboCup soccer scenario 
comprising several interacting agents with different roles and behaviors. In particular, the 
model extracts association rules from the sequences of basic multi-agent actions, showing 
how the constructed rules capture basic soccer strategies such as offensive moves and 
defense schemes. 

More generic models, such as the Agent Academy framework (Mitkas et al., 2004) support 
design and instantiation of agent communities as well as ontology definition. This system 
integrates an agent training procedure that is a mechanism for embedding rule-based 
reasoning capabilities and application-specific knowledge into agents. This is realized by 
means of a data-mining module that processes available behavioral knowledge in order to 
extract decision models for the agents. Another general purpose multi-agent framework is 
(Kaya et al., 2005). It uses association rule mining to estimate agent actions, maintaining an 
historic database of the past agent actions that is processed to extract association rules that 
are, in turn, used to update the action selection strategies of the agents. With a similar intent, 
(Dudek, 2007) proposes the APS (Analysis of Past States) framework, where web browsing 
agents alternate between work phases, during which actions are performed and information 
concerning the user preferences are recorded, and stand-by phases, where the agents are 
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idle and the historic information is mined to extract new user profiles under the form of 
association rules. 

In general, there is an increasing trend of integrating strategic behavioral knowledge into 
agents by mining repositories containing data collected either exogenously to the system 
(e.g. by human exemplification) or by analyzing historic data of the agents themselves. Such 
knowledge extraction mechanism requires tools capable of extracting significant behavioral 
patterns from large data collections. Additionally, the representation of such behavioral 
patterns should allow cross interpretability of the data mining results both from the 
machine and the user perspectives. We advise that association rules can play a key role in 
this sense, serving two key purposes: as knowledge representation means they provide 
humans with insight into data collections; as an action rule base, they can be used to supply 
virtual agents with procedural and high-level knowledge that can effectively be exploited 
for increasing the agents' performance. In particular, we will focus on how the introduction 
of ontologies in the association mining process can successfully improve the interpretability 
of the association analysis 7 s results both from the point of view of the user and from the 
agents' perspective. 

2.3 Ontology 

The ontology may be defined as the study of being as such. The construction of the word itself 
has remote origins and may be seen as an attempt to modernize the classical debate about 
metaphysics. One of the first exponent of this new "discipline" was Aristotle that, in 
(Aristotle, 350 B.C.), presents a first work that can be resembled to an "ontology". He 
provides a list of categories that represent an inventory of what there is, in terms of the most 
general kinds of entities. Later on, the Latin term "ontologia" appeared for the first time in 
the circles of German protestants around the 1600 (Gockelin, 1980) (Lorhard, 1613), while 
the first English appearance goes back to 1721 in the Baileys dictionary in which it was 
defined as "an Account of being in the Abstract". The first use of the term ontology in the 
computer and information science literature occurs instead in 1967, in a work on the 
foundations of data modeling by S. H. Mealy (Mealy, 1967). 

From the scientific point of view, the ontology is, in its first approximation, a table of 
categories in which every type of entity is captured by some node within a hierarchical tree. 
While the philosopher-ontologist has only the goal of establishing the truth about the 
domain in question, in the world of the information systems the ontology is a software (or 
formal language) designed with a specific set of uses and computational environments. In 
this field, the ontologies are rapidly developing (from 1990 up to now) thanks to their focus 
on classification and their abilities in representing the domains, in sharing the knowledge 
and in keeping separate the domain knowledge from the operational one. Furthermore, they 
play a key role in the Semantic Web by providing a source of shared and precisely defined 
terms that can be used for describing the resources. Reasoning over such descriptions is 
essential for accessibility purposes, automating processes and discovering new knowledge. 
One of the modern ontology definition given by (Maedche, 2003) is reported below. An 
ontology O is defined as 

O = {C, R, AO), 

where: 

1. C is a set whose elements are called concepts. 
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2. R cz (C x C) is a set whose elements are called relations. For r = (c\, ci) e R, one may 
write r(ci) = C2. 

3. A is a set of axioms on O. 

To cope with the lexical level, the notion of a lexicon is introduced. For an ontology 
structure O = {C, R, A } a lexicon L is defined as 

L = {LS I*, F, G}, 

where: 

1. L c is a set whose elements are called lexical entries for concepts. 

2. L R is a set whose elements are called lexical entries for relations. 

3. F cz L c x C is a reference for concepts such that 

F(/ c ) = |cgC: (l c ,c) e F] for all l c e LS 

F-i(c) = {/ c e Lc : (/ C/ c) e F} for all c e C. 

4. G cz L R x R is a reference for concepts such that 

G(l R ) = {reR: (l R ,r) e G} for all l R e L* 

G-i(r) = {l R g L*: (/ R ,r) e G} for all r e R. 

We note that a hierarchical structure can be explicitly defined in terms of R. 

Finally, the mapping from elements in the lexicon to elements in the ontology structure 

corresponds to the notion of an interpretation in declarative semantics (Genesereth and 

Nilsson, 1987). Such a (semantic) interpretation associates elements of a language to 

elements of a conceptualisation. Based on the above definitions, an ontology can be formally 

defined as the couple <0, L> where O is an ontology structure and L is a corresponding 

lexicon. The ontology structure O plays the role of an explicit specification of a 

conceptualisation of some domain while the lexicon L provides the agreed vocabulary to 

communicate about this conceptualisation. 

The following example may be useful for explaining the above definitions. 

Let O = {C, R, A } be an ontology structure such that C = {c\, c^\ and R = {r}, where r(c\) = c% 

Suppose that A = 0. Also, let L = {L c , L R , F, C} be a corresponding lexicon such that L c = 

{Mouse, Input _device}, L R = {is_a}, F (Mouse) = C\, F (Input _device) = ci and G(is_a) = r. 

Fig. 2 depicts the elementary ontology described above. The top of the figure shows the 

ontology structure O. It consists of two concepts c\, ci and one relation r that relates them. 

This corresponds to a conceptualisation of a domain of interest without any lexical 

reference. The latter is provided by the lexicon L depicted at the bottom of the figure. L 

provides the lexical references for O by means of F and G. F and G map lexical reference 

strings to the concepts and relations defined in O, respectively. For instance, for r e R one 

may consider using G' 1 ^) to get the lexical reference, i.e. "is_a" , corresponding to r and vice 

versa. 

The Web Ontology Language (OWL) is the current standard provided by the World Wide 

Web Consortium (W3C) for defining and instantiating Web ontologies. An OWL ontology 

can include descriptions of classes, properties and their instances. The OWL language 

provides three increasingly expressive sublanguages: 

• OWL Lite supports a classification hierarchy and simple constraint features. 



180 



Advances in Robotics, Automation and Control 



OWL DL is more expressive than OWL Lite without losing computational completeness 
(all entailments are guaranteed to be computed) and decidability (all computations will 
finish in finite time) of reasoning systems. Its name is due to its correspondence with 
description logics (Baader, 2003), a field of research that has studied a particular 
decidable fragment of first order logic. 
OWL Full is the most expressive with no computational guarantees. 




Fig. 2. Graphical representation of an ontology. 

We focus mainly on the first two variants of OWL because OWL-Full has a nonstandard 
semantics that makes the language undecidable and therefore difficult to implement. OWL 
comes with several syntaxes, all of which are rather verbose. Hence, in this chapter we use 
the standard DL syntax. For a full DL syntax description, please refer to (Baader, 2003). 
The main building blocks of an OWL ontology are (i) concepts (or classes), representing sets 
of objects, (ii) roles (or properties), representing relationships between objects, and (iii) 
individuals, representing specific objects. Furthermore they are composed of two parts: 
intensional and extensional. The former part consists of a TBox and an RBox, and contains 
knowledge about concepts (i.e. classes) and the complex relations between them (i.e. roles). 
The latter one consists of an ABox, containing knowledge about entities and the way they are 
related to the classes as well as roles from the intensional part. In the following is an 
example of DL formalization (according to the syntax defined in (Baader, 2003)) related to 
the previous ontology fragment. We add a property specifying the number of keys of each 
input device 

TBox = ((Mouse cz Input _device) n (Input _device cz Thing) 

n (=2 hasKeysNumber) cz Input jievice 
n (V hasKeysNumber. integer)). 

All the possible ontology instances constitute the ABox which is interlinked with the 
intensional knowledge. An example of ABox related to the previous TBox is the following: 

ABox = {Logitech_device : Input jievice, 

hasKeysNumber = 88, 

Optical _IBM_Mouse : Mouse, 

hasKeysNumber = 2} 
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Given such an ontology, the OWL formal semantics specifies how to derive its logical 
consequences, i.e. facts not literally present in the ontology, but entailed by the semantics. 
The semantics of OWL DL is fairly standard by DL standards. An example is in Fig. 3. 
An interpretation I = (A 1 , • *) is a tuple where A 1 , the domain of discourse, is the union of two 
disjoint sets A<j l , (the object domain) and Ad 1 (the data domain), and I is the interpretation 
function that gives meaning to the entities defined in the ontology. I maps each OWL class C 
to a subset O cz Aq 1 , each object property Pobj to a binary relation P^bj c: (Aq 1 x Aq 1 ), each 



datatype property Puata to a binary relation Phata cz (Aq 
be found in the OWL W3C Recommendation 1 . 



fx 



Ad 1 ). The complete definition can 




Individuals i 1 eA 1 

Logitech _<tevke 

Optk(il_IBM_Mouse 

Concepts C J c J 2 

Thing 

Input_device 

Mouse 

Roles F M( c(4 ^^ 
hasKeysNumber 



Fig. 3. Example of well defined (model theoretic) semantics. 

Concepts and relations represent the explicit knowledge of an ontology, nevertheless 
another kind of knowledge (i.e. the implicit knowledge) can be "deduced" starting from the 
known facts. Its existence is implied by or inferred from observable behavior or performance 
by using the rule inference mechanism. Rules are an extension of the logical core formalism, 
which can still be interpreted logically. The simplest variant of such rules are expressions of 
the form C=> D where C and D are concepts. The meaning of such a rule is "if an individual 
is proved to be an instance of C, then derive that it is also an instance of D" '. Operationally, 
the semantics of a finite set R of rules can be described by a forward reasoning process. 
Starting with an initial knowledge base K, a series of knowledge bases K(°), KM, ... is 
constructed, where K(°) = K and K( i+1 ) is obtained from K® by adding a new assertion D(a) 
whenever R contains a rule C=> D such that K® <C(a) holds, but K® does not contain D(a). 
Semantic Web Rule Language (SWRL) is the W3C proposal for the rule language based on a 
combination of the OWL DL and OWL Lite sublanguages with the Unary/ Binary Datalog 
RuleML sublanguages of the Rule Markup Language 2 . SWRL allows users to write Horn- 
like rules that can be expressed in terms of OWL concepts and that can reason about OWL 
individuals (Horrocks, 2004). According to the SWRL syntax and referring to the previous 
example of TBox and ABox, the inference rule for deducing that an ontology element is an 
instance of Mouse stated that is an Input_device and has 2 keys, is reported below: 



Vx,y Input _device(?x) a hasKeysNumber(lx,ly) a equalTo(?y, 2) ■ 
where equalTo(., .) is a SWRL built-in function. 



► Mouse(?x), 



1 The W3C webpage: http://www.w3c.org/. 

2 Rule Markup Language is specified at www.ruleml.org. 
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3. The system 

This section describes in detail our framework and the process for extracting multi-level 
association rules by using the ontological support (Bellandi et aL, 2008). 
Our scenario consists of the set of components shown in Fig. 4. 



User Interface 




Ontology 
Query Engine 



Fig. 4. Architecture of the system. 

As in the most common system architectures, our framework is composed of a set of 
"modules" and a set of repositories. There is a repository for the metadata, i.e. the 
ontologies, and a repository for the transactions, i.e. the DataBase (DB). The former 
repository contains the Domain Ontology (DO) and the Rule Ontology (RO). The DO 
describes the domain of interest, i.e. all the items that compose the transactions and that are 
stored in the DB. The RO, instead, is used for storing the association rules extracted during 
the process. It is different from the DO because it models the structure of an association rule 
according to its definition (see Section 2.1). The DO is the description model for the 
collection of data and is not designed for containing actual instances. The RO, instead, is 
used both for describing and containing the inference rules. 

The User Interface provides mechanisms for interacting with the system and for visualizing 
the results. The user can provide the system with two types of input: a set of user constraints 
for specifying what items must be considered in the discovering process, and another one 
for filtering the extracted association rules. The user specifies constraints by using the syntax 
presented in the Table 1. It includes both pruning constraints, used to filter a set of non- 
interesting items, and abstraction constraints, which permit a generalization of the item on a 
concept of the ontology. We informally describe each constraint by means of an example. 
By using pruning constraints, one can specify a set of items which can be excluded from the 
input transactions set, and, as a consequence, from the extracted rules. This kind of 
constraints refers either to a single item, or to an ontology concept, and can include a 
condition expressed on a set of ontology properties. There exist two kinds of pruning 
constraints: 
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X is the set of items (ii/ia, ---in £ T). 

C is the set of the concepts of the ontology (ci,ca, ...c n £ C). 

V c is the set of the properties of the concept c £ C (pi,p2, -Pn GP C ). 

co7id c is a Description Logic expression. 

ALL represents all the instances defined in the ontology. 

A constraint is defined on I\ C and Vc in the following form: 

1. Pruning Constraints, A pruning constraint is of one of the following forms: 

(a) prurte(e), where e £ TlJC U {ALL}. 

(b) prune con <t c (c), where c £ C U {ALL}. 

2. Abstraction Constraints, An abstraction constraint is of one of the following 
forms: 

(a) ahstract(e, c), where e £ T U C, c £ C and c is a super-concept of e. 

(b) abstract CO n d cl (cijCa) where c\ £ CU{ALL\, 02 £ C and C2 is a super-concept 
of Ci. 

(c) abstract l conde (e) : where e £ I U C U {ALL} : and I is a non-negative integer 
indicating the level of the hierarchy; cond can be unspecified. 



Table 1. Syntax of the Pruning and Abstraction constraints. 

• The constraint prune(e) excludes a single item or all the items belonging to an ontology 
concept expressed by "e". As one could expect, removing an ontology concept c e C 
also implies excluding all the items belonging to the descendent concepts of c. 

• The constraint prune CO nd(c) is similar to prune(e), except for uninteresting items that are 
selected according to a condition defined on a set of properties of c. The condition can 
also refer to a property of an ancestor of c. 

For example, let us suppose to have a simple ontology modelling all possible soccer actions 
performed by soccer agents. Every item represents an action and has a property 
hasPassesNumber to indicate the number of passes of an action. The ontology has the form 
of a taxonomy organized according to the kind of action (e.g. Action, Goal Action, Non- 
GoalAction, Corner Action, etc.). The predicates prwne(Non-GoalAction) and 
prunehasPassesNumber>5(Goa\Action) allow to prune all the " Non-Goal Action" actions and all the 
" Goal Action" actions with a number of passes greater than 5, from the input transactions. 
Notice that hasPassesNumber is a property of the upper concept Action, while Non- 
GoalAction and GoalAction are sub-concepts of Action. 

The Abstraction constraints enable to explore different levels of the ontology concepts. The 
generalization to a predefined level of the hierarchy increases the support of association 
rules and, consequently, avoids the discovery of a massive quantity of useless rules, 
especially in case of sparse data. More interesting is the exploitation of rules in which items 
are extracted at different levels of the hierarchy. The system offers three different predicates: 

• The predicate abstract(e, c) generalizes a single item e (or all the items belonging to the 
concept e) to the super-concept c. 

• The predicate abstract con d< c i>{ci, C2) is similar to the previous one, but it generalizes items 
according to a condition (cond<ci>) defined on a set of properties of the concept ci. Only 
items satisfying this condition are abstracted to the super-concept C2. As for prune con d(c), 
this condition can also refer to a property of an ancestor of ci. 

• Finally, the abstract 1 ' C ond<e>(e) generalizes every element satisfying the condition cond<e> 
to the super-concept placed at level 1 of the hierarchy. In this case, e can be either a 
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single item or a concept. If the condition is not specified, all the items belonging to e are 

generalized to the super-concepts at level 1. 
The symbol " ALL" in Table 1 can be used to select all the items defined in the ontology. 
Suppose that the ontology provides also a description of all the soccer field areas, organized 
as an hierarchy; this permits to specify in which soccer field area each action is performed. 
Lets assume that LittlePenaltyArea is a sub-concept of Penalty Area. The predicate 
atorac£(LittlePenaltyArea, Penalty Area) generalizes all the actions performed in the "Little 
Penalty Area" to a more generic "Penalty Area". It is equivalent to the predicate 
atorac£ 5 (LittlePenaltyArea), if Penalty Area is placed at level 5 of the ontology. 
The set of constraints Vd is therefore a conjunction of pruning and abstraction constraints. 
The Interpretation Module in Fig. 4 translates the requests of an user into a set of formal 
constrains (defined on the DO) so that they can be supplied to the Ontology Query Engine 
by means of a suitable query language. In our system we used the Sesame RDF Query 
Language - SeRQL (Broekstra, 2004). 

The Ontology Query Engine interacts with the DO by performing some queries and 
retrieving the desired set of items (IS). This module uses a particular ontology query 
language according to the ontology management system that supports the operations. 
The output is used by the DB Query Engine for retrieving the DB transactions that contain 
the items specified in IS. 

The Association Rule Extraction Module contains an implementation of the Apriori 
algorithm that computes the set of association rules (AR). Such a rules are then stored in the 
RO. Finally, the Post-filter Module evaluates the post filtering constraints for selecting the 
most relevant association rules w.r.t. the user needs. Since the extracted rules are stored in 
the OR, this module interacts with the Ontology Query Engine for retrieving the correct 
items by means of directed queries. The post-filter module interacts with the GUI for 
receiving the SERQL queries and for presenting the output rules. It is important to notice 
that the constraints are conceived not only on the structure of the rules, but also on their 
semantics, since the domain ontology provides a mean to express the semantics of single 
items belonging to the rules. For example, let us consider the case in which the ontology 
models all the possible soccer actions performed by soccer agents. Association rules can be 
mined to highlight relations among the actions performed by all players. Given a set of 
output rules, we are able to focus only on those in which the antecedent contains at least one 
kick action made by a player having a particular role. In this case, the semantics of the item 
"kick" and "player role" are encoded in the domain ontology. 

4. Discovering strategic behaviors by association rules mining 

In this section we describe how the association rule mining system presented above can be 
instantiated in the context of behavior discovery in multi-agents system (in Section 4.1), and 
we describe (in Section 4.2) an application example in the field of simulated robotic soccer. 

4.1 Multi-agent planning with ontology-based association rules 

Earlier in the chapter, we have discussed the increasing trend of behavior discovery in 
multi-agent systems. Here, we focus on the role of association rule mining as a means for 
extracting such strategic action patterns: in particular, we point out the capital importance of 
an ontology based approach for extracting high-level planning knowledge that can 
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understood both by the human users as well as by the software agents. For this reason, we 
present an ontology-based planning scheme for intelligent computational entities. 
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ONTOLOGY | 
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Fig. 5. Behavior mining with association rules in a multi-agent system 

Consider, for instance the scenario in Fig. 5: a set of reference agents, either human or 
artificial, is interacting within an environment, each of them pursuing a particular goal 
based on its internally defined policy. The configurations of the system, including the courses 
of actions of the agents as well as characteristic information (e.g. spatial location, battery 
level and signal strength in radio agents) are monitored and logged to a database (DB in Fig. 
5). After a sufficient amount of time, the database will contain massive amounts of flat 
information referring to the development of the agents' behavior in time, as well as the 
strategies used to pursue their goals. Our work aims at analyzing such large data repository 
to extract a knowledge base (KB in Fig. 5) holding higher level information content that can 
be used to refine the planning strategies of the replicating intelligent agents in Fig. 5. 
Consider, for instance, the elements in Fig. 5 as being players from two soccer teams: by 
monitoring their actions during the whole match we can log interesting information, such as 
player location, speed, stamina, as well as events of interest, e.g. kick, pass, header, slide, 
dash, etc. Then, by applying association analysis, we can extract a set of association rules 
that describes the most likely decisions to be taken for the most frequent configurations of 
the game. For instance, the knowledge base can end up containing rules such as 

IF LeftPlayerlO. location IS RightQuarterMidfield AND leftPlayerlO.pass AND 
LeftPlayerl 1 . location IS RightPenaltyArea 
THEN leftPlayerll.dash; 

which describes a typical soccer attack strategy, that is "when player 10 is on the attack and 
passes the ball, then player 11 should run for the pass in the penalty area". The extracted 
rules are, in turn, used to populate our knowledge repository KB, which includes a 
description of our reference world given by the domain ontology. 

For the particular problem at hand, i.e. soccer matches, the ontology can include, for 
instance, a taxonomy of the areas of the field, a description of the players (e.g. in terms of 
physical/ physiological parameters such as height, weight, stamina, speed, etc.) as well as a 
multi-layered description of the actions and events (e.g. the pass action that can be further 
differentiated in long and short pass, heal pass, etc.). 
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The novelty of our approach lies in the full integration between the ontology and the 
association analysis: the ontological description can be used to navigate the results of the 
association analysis at a given level of abstraction by imposing ontology-based constraints 
to filter the association rules in the KB. More formally, this is obtained by applying the rule 
ontology population scheme in Fig. 4: by exploiting the knowledge representation in the 
domain ontology we perform queries that filter the association rules at particular level of 
abstraction. The resulting rules serve to populate the rule ontology that is in turn used for 
the decision making process. 

Association rules filtering is of capital importance when approaching behavior mining in a 
multi-agent system, due to the explosive rule base complexity that may arise as a result of 
the association analysis. The interaction of multiple agents, in fact, generates a massive 
amount of data that populates the DB; as a consequence the traditional association analysis 
can generate a large number of rules addressing the behavior of several agents. Thus, a 
pruning technique is needed to reduce the cardinality of the association rules by filtering the 
results of the association analysis to obtain personalized views of the discovered procedural 
knowledge. In our approach this mechanism is implemented by means of ontology-based 
filtering. For instance, if we are interested in knowledge regarding heal pass actions 
performed in opponent's team area, we apply the ontological definitions of heal pass and 
opponent's area to the association rules in the knowledge repository. As a result only those 
rules satisfying the constraints are retained and used to plan the next decision. This 
approach offers a two-fold advantage: on the one hand, it allows human users to selectively 
tune their view on the extracted knowledge so that their analysis of the data can be more 
tightly focused. On the other hand, such an approach is fully exploitable by intelligent 
software agents to obtain behavioral patterns that are tailored to the particular decision 
scenario they are addressing. The ontological description of the world can, in fact, be easily 
coded into the software agent. The agent itself can then use it to gather knowledge from the 
mare-magnum of the association rule base at the required descriptive level (i.e. abstraction 
constraints) or for a particular ontological relation (i.e. property constraints). By this means, 
we can define intelligent software agents that exploit their world description, i.e. the 
ontology, to learn useful behavioral patterns, i.e. the ontology-filtered rules, from a vast 
amount of flat data. 

Going back to our soccer example, we can interpret the replicating agents in Fig. 5 as being 
players in a virtual soccer match: the rule base extracted from the actual soccer players is 
accessed by each replicating agent that applies its ontology-based filter to extract those 
behavioral patterns that best fit the current game configuration. Then, the planning module 
of the replicating agent determines the next action (or sequence of actions) based on the 
retrieved decision rules (see Fig. 6 ). In a sense, the replicating agent is learning a behavioral 
pattern by analyzing the most frequent configurations generated by the actual agents and 
selecting only those that fits best its role and its personal view of the world. Notice that, in 
general, nothing prevents the real agents to be the same as the replicating agents; hence, by 
devising smart association rule filters, an agent can be made to learn from the history of its 
past actions. 

Fig. 6 depicts a prototypical planning module for a replicating agent that is based on the 
proposed ontology-driven behavior mining model. The agent builds the current 
configuration descriptor by gathering information from the environment, the proprioceptive 
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sensors as well as from its internal state. This information is used to determine the 
preconditions for the decision rules and is exploited by the planner to elaborate the next 
course of actions. In order to do so, the agent planner queries its knowledge base at the 
required level of abstraction and obtains a set of decision rules that match the current 
system configuration. Based on these rules, the planner determines the next actions that are 
forwarded to the actuator sub-system. 
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Fig. 6. Agent planner based on the ontology-driven behavior mining model 

The simple soccer scenario that we have adopted to introduce the behavior mining scheme 
is not far from a real-world application of the model. Several soccer teams, in fact, have 
started monitoring their players with sensors both during training sessions and during 
games (e.g. see the MilanLab 3 structure of AC Milan). Such information, if accessible, can in 
principle be used to extract strategic action patterns that can be exploited, for instance, to 
devise intelligent software agents for virtual soccer matches, such as in the Robocup league. 
Unfortunately, such data has not been disclosed, yet: hence, in the next section, we present a 
behavior extraction example based on the freely available logs from the Simulated Soccer 
League. 

4.2 Extracting strategic behaviors from robot soccer data: an example 

In this paragraph we provide a schematic example describing the steps of the analysis 
process. The example is based on data collected from the RoboCup simulated league. 
RoboCup is an international initiative to foster artificial intelligence and intelligent 
robotics by providing a standard test bed where several technologies can be integrated 
and examined. The RoboCup simulated league consists of teams of autonomous software 
agents competing against one another in soccer games. Logs were taken from each 
simulation game consisting of about 6000 simulation cycles. Each cycle contains 
information about player location and orientation, ball location, ball possessor, game 
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score and play modes such as throw-ins, goal-kicks and so on. Since our aim is to "mine" 
the behaviour of a team in the soccer RoboCup league in the 2004 edition 4 , we have 
collected all the log events related to the match played by the winner team, focusing on 
position and action information. 
Stepl: The ontology definition. 

We defined an ontology for describing the entities of the soccer domain, focusing on the 
variables actually contained in the collected data. We built the ontology by creating a 
taxonomy of concepts, adding interesting data as well as object properties according to the 
target of our analysis. 

In Fig. 7 is reported the ontology of the main concepts (superclasses) and their data and 
object properties. For our purposes it was necessary to model the concepts Player, Kick, Body, 
Ball, PlayerRole and FootballFieldArea. Concerning the kick action, it is interesting to know the 
actor and the receiver (2 players), where it is directed (the field area) as well as the body part 
that has produced it. The corresponding object properties are madeBy, towards, 
hasDestinationArea, madeByllsing. 



Kick 


madeByUsing Distance* Body 


hasDestinationArea | Instance* | FootballFieldArea 


towards 


Instance* 


Player 


lnadeBy 


Instance 111 


Player 





hasBallPositionX 



liasBallPositionY 



isInFieldArea Instance* FootballFieldArea 



Player 
ha sPlayerPositionX 



ha sPlayerPositionY 



hasNuinber 



Any* 



isInFieldArea Instance* FootballFieldArea 



belongsTo 



FootballFieldArea 



partOffield String* 



PlayerRole 



Fig. 7. The domain ontology for the soccer domain: the super-concepts. 

For a player is interesting to know its role and its position. The latter information is of 
interest also for the ball concept. 

For the concepts Body, FootballFieldArea and PlayerRole we realized three taxonomies of sub- 
concepts (Fig. 8, Fig. 9 and Fig. 10 respectively), that in the following will be used to express 
AR constraints. 



[ http://staff.science.uva.nl/~jellekok/robocup/rc04/ 
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Fig. 8. The Body taxonomy. 




Fig. 9. The FootballFieldArea taxonomy. 
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PlayerRole 
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CenheForward 
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liasOntsidePosition String 3 
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hasInsidePosition Shing : 



Fig. 10. The PlayerRole taxonomy. 

The information related to all the previous ontological entities, was not explicitly 
represented in the DB. Starting from the information contained in the DB, we inferred the 
new information according to our ontology entities. For instance, in the FootBallFieldArea 
taxonomy, we split the soccer field into 24 disjointed parts, that are used to characterize 
players and ball positions, as showed in figure Fig. 11. 
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Left Side of The Soccer Field 
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Fig. 11. Parts of the soccer field. 
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Starting from the coordinates of each position, inference rules are used for identifying each 
part of the soccer field. The SWRL rules have the following form: 

Vx,y,z Player(?z) a hasPlayerPositionX(?z / ?x) a hasPlayerPositionY(?z / ?y) a 
GreaterOrEqualThan(?x, -52.5) a LessOrEqualThan( ?x, -47) a 
GreaterThan( ?y, -9.2) a LessThan( ?y, 0) 
-> isInFieldArea(?z, LLA_L). 

This rule states that a certain player (identified by the variable z), belonging to the Left team, 
is in the left part of the little area (LLA_L), where LLA_L is an individual of the LeftLittleArea 
ontology class. Similar rules have been used for deducting all the other ontology entities. 
According to the architectural description in Section 3, we introduce an ontology for storing 
the ARs, that is shown below in Fig. 12. 
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Fig. 12. The Rule Ontology. 

This ontology models the structure of an association rule. The concept RDA identifies a rule: 

it has the reference to the confidence and the support, as well as the link (l:n relation) to 

concepts Antecedent and Consequent, that are, in turn, used to model the left and right hand 

side of a rule. 

Step2: Data collection and preparation. 

Raw data consists of about 65.000 transactions and 108 attributes. The structure of each one 

of the 22 players is reported in Table 2. Moreover, we have some actual attributes related to 



Attribute 


Type 


Description 


pos_x 


real 


X Coordinate of a player position in the football field 


pos_y 


real 


Y Coordinate of a player position in the football field 


actions Jack 


{0,1} 


1 if actual action is equal to kick, otherwise 


actions_dash 


Ri} 


1 if actual action is equal to dash, otherwise 


actions Jrurn 


Ri} 


1 if actual action is equal to turn, otherwise 


actions _catch 


{0,1} 


1 if actual action is equal to catch, otherwise 


actions jmove 


{0,1} 


1 if actual action is equal to move, otherwise 


calculated ^possessor 


{0,1} 


1 if player has ball, otherwise 



Table 2. Player Attributes. 



192 Advances in Robotics, Automation and Control 

the ball state, such as the coordinate of the ball in the football field (ball_position), the 

distance of the ball from a goal (ball_goal_distance) and the possessor team of the ball 

(ball_possessor). 

The rule extraction algorithm is not applied directly to the collected data described above, 

whereas such information is abstracted or filtered in agreement with the user specified 

constraints. For example a player position, expressed by x-y coordinates, can be abstracted 

to whatever field zone defined by the ontology. 

We selected all the actions where the team under analysis is in " offensive state". To 

determine an "offensive state", we abstract the balljpositiori attribute to an ontology high 

level concept considering only those actions in which the ball is into a specific part of the 

football field, for example PenaltyArea, or simply in the opponent midfield. Then, we filtered 

the data considering only the actions where at least one player executes a kick. Finally, we 

observed the behavior of the forward players (e.g. with numbers from 9 to 11) related to the 

opposite defenders (e.g. with numbers from 2 to 5). 

It is important to notice that all the data transformations introduced above are expressed by 

means of constraints defined on the basis of a well-defined domain ontology, and 

automatically executed by the software interpreting the results provided by the ontology. 

Step3: The constraints definition. 

Our simple experimentation concerns two queries. In the first case, the objective is to find 

frequent itemsets involving "kicking players" and actions performed in the right middle 

soccer field. In the second case, we are interested in these specific kicking players: PlayerL9, 

PlayerLlO, PlayerLll, PlayerR2, PlayerR3 and PlayerR4. By using our constraint language, 

an user can formulate the first query in the following way: 

queryl = (prune (bdongsTo = R)(Player) 

A prune (hasNumber > 5 orhasNumber = 1) (Player) 

a prune (b eion g sTo= 'L'j(Player) 

a prune (hasNumber < 9) (Player) 
a prune dsinFieid = Ri g htPart)Q?layer) 
a prune ( ma deBy-i = 'null') (Player) 
a afrs£n?c£ 3 (FootBallFieldArea)) 

In queryl, the first four clauses are used for pruning all the players PlayerRl, PlayerR5, 
PlayerR6, PlayerR7, PlayerRS, PlayerR9, PlayerRlO, PlayerRll, PlayerU, PlayerU., PlayerU, 
PlayerL4, PlayerU), PlayerL6, PlayerU and PlayerL8. The 5 th clause permits to prune all the 
players performing actions in the left part of the soccer field, while the 6 th one permits to 
prune all the players not kicking (by using the inverse property of madeBy). The last clause 
abstracts all the positions in the filed to level 3 of the FootBallFielArea taxonomy. 
The second example is defined by the following query: 

queryl = (prune (bdongsTo = 'R' (Player) 

A prune (hasNumber > 5 or hasNumber = 1) (Player) 

a prune(beion g sTo= T')(Player) 
a prune (hasNumber < 9) (Player) 
a prune (MnFieid = Ri g htPart)Q?layer) 

A prune ( m adeBy-l = LI or madeBy-1 = L2 or madeBy-1 = L3 or 
madeBy-1 - L4 or madeBy-1 = L5 or madeBy-1 - L6 or 
madeBy-1 = U or madeBy-1 = L8) (Player) 
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A prime (madeBy-1 = Rl or madeBy-1 =R5 or madeBy-1 = R6 or 
madeBy-1 = R7 or madeBy-1 =R8 or madeBy-1 = R9 or 
madeBy-1 = RIO or madeBy-1 = Rll) (Player) 

a abstract 3 (FootBallField Area)). 
Query 2 is very similar to query 1, but instead of selecting all the kicking players, it retrieves 
only those items in which either PlayerL9, PlayerLlO, PlayerLll, PlayerRl, PlayerR3 or 
PlayerR4 performs the kick. 
Step4: Extracting the Behavior Patterns. 

Once that the ontology-based query has gone through all the steps discussed above, we 
obtain a populated rule ontology specifying a set of association rules that describes the 
behavioral patterns of interest. 

As an example, let us focus on defensive strategies by centering our analysis on the behavior 
of the defenders D = {PlayerRl, PlayerRl, PlayerR3, PlayerR4} in the right-side team with 
respect to the forwards F = {PlayerL9, PlayerLlO, PlayerLll} of the left team. To study the 
positioning of the players at a sufficient level of detail, we abstract to the bottommost layer 
of the taxonomy in Fig. 9 (see Fig. 11 for a visual representation of the field areas). 
Moreover, to quantize actions to relevant time instants we filter out those transactions where 
the ball is not kicked by any player. If we are interested in discovering knowledge 
regulating the in-field positioning of the defenders, we can query for relevant association 
rules regarding the players position on the field. An example of high-confidence ontological 
rules produced by this association analysis is: 

a. PlayerL10_RMQ_R / PlayerL9_LMQ_R, PlayerR2_QLLS_R, PlayerR4_RMQ_R -> 
PlayerR3_LMQ_R; 

b. PlayerL10_RMQ_R / PlayerLll_LMQ_R / PlayerR2_LMQ_R -> PlayerR4_RMQ_R; 

c. PlayerLll_LPA_R / PlayerL9_LPA_R, PlayerR3_LPA_R PlayerR5_RPA_R / -> 
PlayerR2_LPA_R; 

d. PlayerL10_RMF_R / PlayerR5_MRLS_R -> PlayerR4_RMF_R. 

Rule (a), for instance, describes the typical defense scheme where a player, i.e. PlayerLlO, is 
controlled by the defender PlayerR4, while a second attacker, i.e. PlayerL9, has to be 
defended by PlayerR3 which, indeed, has to position in the same area of the field of PlayerL9. 
Notice that the rule states that a third defender, i.e. PlayerRl, is already holding a position 
and should not defend on PlayerL9. Similarly, rule (b) requires PlayerR4 to defend on 
PlayerLlO while PlayerRl is on PlayerLll. These kinds of rules can be used by those agents 
implementing the defender players to determine their position on the field in standard 
defense schemes. Rule (c) describes a dangerous situation in which two forwards, i.e. 
PlayerLll and PlayerL9, are in the left side of the penalty area: PlayerR3 is defending on both 
players while PlayerR4 is on the right side of the penalty. Hence, a smart defense strategy 
requires PlayerRl to help PlayerR3 on the left side. On the other hand, rule (d) describes a 
typical pattern where PlayerR5 should keep its position, not defending on PlayerLlO, while 
PlayerR4 should localize in the same field position as PlayerLlO. 

If we are interested in discovering action patterns correlated with the defense schemes, we 
can let action concepts into the itemset together with the players' positions. An example of 
rules that we obtained from the simulated soccer data is: 

e. PlayerR2_LMQ_R_dash, PlayerR3_LMQ_R_dash -> PlayerR4_RMQ_R_dash; 

f. PlayerL9_RMQ_R_dash -> PlayerR4_RMQ_R_dash. 

The former rule describes a typical joint defensive movement, that is: when two defenders 
located in the same field area dash, then neighbor players, i.e. PlayerR4, should follow them. 
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The latter rule, on the other hand, states probably the most typical defensive movement: if a 

forward PlayerL9 dashes in the field area of a defender PlayerR4, then PlayerR4 should 

follow him. 

The same approach that has lead us to discover defensive patterns can be used to extract 

knowledge concerning attack strategies. For instance, let us study first the positioning on the 

field for the forwards F = {FlayerL9, FlayerhlO, PlayerLll} of the left team. By querying the 

rule ontology we obtain rules such as: 

g. PlayerLll_QLLS_R, PlayerL9_LMQ_R, PlayerR4_LMQ_R --> PlayerL10_RMQ_R; 

h. PlayerLlOJiasBall, PlayerL9_RMQ_R --> PlayerLll _LMQ_R; 

i. PlayerL10_RPA_R, PlayerLlOJiasBall PlayerL9_RPA_R / PlayerR4_RPA_R -> 

PlayerLll_LPA_R. 
Rule (g) describes a basic attack strategy where the forward PlayerLll is off-center on the left 
lateral side while PlayerL9 is controlled by the defender PlayerR4 on the center-left; 
therefore, the third forward PlayerLlO should chase the space on the center-right area. Rule 
(h) identifies another fundamental, yet very simple, attack strategy that is: if PlayerLlO has 
the ball and is about to kick (notice that we have filtered kick-only timeframes) then the two 
other forwards, i.e. PlayerL9 and PlayerLll, should position outside the penalty area, 
respectively on the right and on the left, waiting for a pass or a rebound. Rule (i), on the 
other hand, depicts a clear scoring chance where PlayerLlO is holding the ball in the right 
penalty area and PlayerL9 is defended by PlayerR4 on the same position, hence PlayerLll 
should chase the space, and a possible rebound or pass, on the left of the penalty. 
Likewise with defensive strategies we can query for attack action patterns by including 
action concepts in the itemset. Notice that there are two approaches for achieving this: the 
former, that was already shown in rules (e)-(f) considers items as a concatenation of actions 
and positions; the latter considers actions and positions separately. An example of attack 
strategies is: 

;. PlayerL9_LMQ_R_dash -> PlayerL10_RMQ_R_dash; 

k. PlayerLlOJiasBall PlayerL9_RMQ_R -> PlayerLlO Jack, PlayerLll _LMQ_R; 
I PlayerLUJiasBall PlayerL9JLMQ_R, PlayerR3JLMQ_R -> PlayerLlO JRMQJR, 

PlayerLll Jack. 
Rule (j) is, again, an example of the first approach: it simply states that PlayerLlO, positioned 
on the right of the penalty area, should follow PlayerL9 when he dashes outside the left of 
the penalty area (i.e. joint attack). Rules (k) and (1), on the other hand, are two examples of 
the second approach. The first rule, for instance, states that, if PlayerLlO has the ball and 
PlayerL9 is positioned outside the right of the penalty area, then PlayerLll has to chase the 
space on the left and PlayerLlO should shoot or pass. Taking a closer look at the database 
transactions that falls in the support of rule (k) confirms that, actually, PlayerLlO tries to pass 
or shoot the ball whenever this rule is enabled. Similarly, rule (1) states that PlayerLlO should 
occupy the area on the right of the penalty and PlayerLll should pass or shoot if PlayerL9 is 
defended by PlayerR3 on the left. 

The rules described above are just a small sample of those obtained by the proposed 
ontology-based association analysis on the simulated soccer data. In general, each agent 
implementing a particular role should filter the rules of its interest, that is those containing 
interesting information on the consequent. By exploiting its internally defined ontology, 
each agent can obtain personalized views of the knowledge concealed in the flat data. 
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Typically, an agent could be interested only in association rules whose consequent refers to 
a specific player, e.g. PlayerLlO. By using our constraint language, the agent can express 
such a post-processing constraint in the following way: 

query_post = pn/ne(ha S piayerRef<>PiayerLio) (Consequent). 

Indeed, the value of the retrieved expert knowledge will depend strictly on the quality on 
the world description, i.e. the domain ontology, as well as on the available data, that is to 
say, if we rely solely on the strategies of loser teams, we will most probably obtain also-ran 
agents. 

5. Conclusion 

We have introduced an ontology-based approach for association analysis in the context of 
behavior mining in multi-agent systems. Our proposal is based on the idea of the 
ontological description of the domain as an essential via-point for accessing the expert 
knowledge concealed underneath massive amounts of "flat" data. The introduction of a 
multi-layered and multi-relational representation of the domain allows approaching the 
information content from several, diverse, viewpoints. Within the multi-agent area, this 
approach offers considerable advantages since it allows the agents to gather personalized 
views of the extracted knowledge, represented by means of rule ontologies. By exploiting 
this "relativistic" representation, an agent can dynamically generate and selectively access, 
at the desired level of abstraction, the knowledge that is of higher relevance for its current 
decision-making activity. Besides presenting the clear advantage of offering personalized 
views of the world, which is totally consistent with the multi-agent model, this approach 
relieves the agents from the burden of acquiring, maintaining and mastering a monolithic, 
and encyclopedic, representation of their knowledge. 

The model presented in this chapter tackles the association analysis task with the standard 
static approach where each transaction is considered in isolation and not as part of a 
spatiotemporal trajectory. Sequence mining, on the other hand, studies how to approach the 
problem of finding frequent patterns for trajectory data. An interesting future development 
for our model would be to extend it to sequence mining (Agrawal & Srikant, 1995): in 
particular, this would be of great interest for behavioral pattern mining, since it can 
naturally tackle the problem of planning medium to long term strategies comprising lengthy 
sequences of inter-dependent actions. However, we would like to point out that our model 
already offers a means for processing multiple transactions in a sort of time trajectory. 
Through the domain ontology is possible, in fact, to describe concepts whose definition 
transcends the single transaction. Consider, for instance, the pass concept in the soccer 
ontology: since it requires to specify source and destination of the action, its instantiation 
needs to process multiple transactions to find all the requested information, e.g. the receiver 
of the pass. 
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1. Introduction 

The resolutions of neurons networks training problems by gradient are characterized by 
their noticed inability to escape of local optima [Mich93], [Fabr94] and in a least measure by 
their slowness [Wess92], [Zhan92]. The evolutionist algorithms bring in some domains a big 
number of solutions: practice of networks to variable architecture [With90], automatic 
generation of Booleans neurons networks for the resolution of a class of optimization 
problems [Grua93]. However the effort of research was especially carried on the generation 
and the discreet network training. 

In this chapter, we propose a new algorithm of wavelets networks training, based on 
gradient that requires: 

• A set of training examples: the wavelets networks are parametrables functions, used to 
achieve statistical models from examples (in the case of classification) or of measures (in 
the case of modeling); their parameters are calculated from these examples or couples 
{input, output}. 

• The definition of a cost function that measures the gap between the input of the 
wavelets network and the desired output (in the case of classification) or the measured 
values (in case of modeling) present on the set of training. 

• A minimization algorithm of the cost function. 

• An algorithm of selection of basic function to initialize the network parameters. 

We try then to show the importance of initialization of the network parameters. Since the 
output is non linear in relation to these parameters, the cost function can present local 
minima, and the training algorithms don't give any guarantee to find the global minimum. 
We note that if we have a good initialization, the local minimum problem can be avoided, it 
is sufficient to select the best regressions (the best based on the training data) from a finished 
set of regressors. If the number of regressors is insufficient, not only some local minima 
appear, but also, the global minimum of the cost function doesn't necessarily correspond to 
the values of the searched parameters, it is useless then in this case to put an expensive 
algorithm to look for the global minimum. 

With a good initialization of the network parameters the efficiency of training increases. A 
very important factor that it is necessary to underline is: whatever the chosen algorithm, the 
quality of training wavelets networks is as much better than we have an optimal 
initialization. 
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2. New wavelets networks architecture 

2.1 Presentation 

From a given wavelets network architecture, it is possible to generate a family of 
parametrables functions by the values of the network coefficients (weight, translations, 
dilations). 

The objective of the wavelets networks training phase is to find, among all these functions, 
the one that approaches the most possible regression (Beta function for example). This one is 
unknown (otherwise it would not be necessary to use an approximation by wavelets 
networks); we only know the observed values (values of the regression to which are added 
noise) for several values valued by the input (points of the training set). 
We consider wavelets networks as follows [Belli07]: 

N, N 2 N M N t 

y = 2>y; (x) + 2», 2 (*) +....+X>v (*) + 1>* x k a) 

i=\ i=l i=l k=0 

M N Mw N t 

= XX *>>/(*) + Z fl * x k (2) 

7=1 i=\ k = 

N Mw N t 

= Z *%M*) + zL a k x k (3) 



KU)=1 



with N Mw =Y N i > / = [1,....,^],7=[1,.-,M], x = 1 (4) 

Where y is the network output and x = {xi,X2, ..., xni } the input vector; it is often useful to 
consider, in addition to the wavelets decomposition, that the output can have a linear 
component in relation to the variables: the coefficients ak (k = 0, 1, ... , Ni). 

Ni is the number of selected wavelets for the mother wavelet family X F / . 

The index 1 depends on the wavelet family and the choice of the mother wavelet. 
The network can be considered as constituted of three layers: 

• A first layer with Ni input. 

• A hidden layer constituted by Nm w wavelets of M mothers wavelets each to a wavelet 
family of size Ni. 

• A linear output neuron receiving the pondered wavelets outputs and the linear part. 
This network is illustrated by the figure 1. 

2.2 Description of the procedure of library construction 

The first stage of the training procedure consists in the construction of the Beta library. We 
intend to construct a several mother wavelets families library for the network construction. 
Every wavelet has different dilations following different inputs. 

This choice presents the advantage to enrich the library, and to get a better performance for 
a given wavelets number. The inconvenience introduces by this choice concerns the size of 
the library. A wavelet library having several wavelets families is more voluminous than the 
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one that possesses the same wavelet mother. It implies a more elevated calculation cost 
during the stage of selection. 




Fig. 1. Graphic representation of the new wavelets network architecture 

Nevertheless, using classic algorithms optimization, the selection of wavelets is often shorter 
than the training of the dilations and translations; the supplementary cost introduced by 
different dilations can be therefore acceptable. 
We have a sequence of training formed of N examples distributed in the interval [a, b]. Let 

Ta mother wavelet family, x the variable, ti the translation parameter and di the dilation 

parameter. The wavelet ^ of the family *¥ J having for parameters ti and di is defined as: 

"¥ J i (x) = "¥ J (d i (x-t i )) (5) 

The wavelet library W, generated from the mother wavelet family, is defined as: 

W = { V (rf f («-*,)),</,£ J?; ,* f e J? J = [1....,M]} (6) 

= {W\ ,W N } (7) 

= y(d i (x-t l )), ; ¥ u {d t {x-t t ))} (8) 
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3. New algorithm: MLWNN (Multi-Library Wavelet Neural Network) 

In this paragraph, we propose a new selection wavelets algorithm [Bellil07-a], based on the 
architecture given on the figure lthat permits to make: 

• The initialization of weights, translations and dilations of wavelets networks, 

• The optimization of the network parameters, 

• The construction of the optimal library, 

• The construction of the wavelets networks based on discreet transform. 

The new architecture of wavelets networks founded on several mother wavelets families 
having been defined. Consequently, we can ask the question of construction of a model, 
constituted of wavelets network for a given process. 
The parameters to determine for the construction of the network are: 

• The values to give to the different parameters of the network: structural parameters of 
wavelets, and direct terms. 

• The necessary number of wavelets to reach a wanted performance. 

The essential difficulty resides in the determination of the parameters of the network. 
Because the parameters take discreet values we can make profit to conceive methods of 
wavelets selection in a set (library) of discreet wavelets. The conceived performance 
depends on the initial choice of the wavelets library, as well as a discriminating selection in 
this library. 

3.1 Principle of the algorithm 

The idea is to initialize the network parameters (translations, dilations and weights) with 
values near to the optimal values. Such a task can be achieved by the algorithm "Orthogonal 
Forward Regression (OFR)" based on the algorithm of orthogonalization of Gram - Schmidt 
[Chen89], [Ouss98], [Chen06], [HoOl]. 

Contrary to the OFR algorithm in which the best regressors are first selected [Lin03], 
[Rao04], [Xiao04], [AngrOl], then adjusted to the network, the algorithm presented here 
integrates in every stage the selection and the adjustment. Before every orthogonalization 
with a selected regressor, we apply a summary optimization of the parameters of this one in 
order to bring it closer to the signal. 

Once optimized, this new regressor replaces the old in the library and the orthogonalization 
will be done using the new regressor. 
We describe this principle below in detail. 

3.2 Description of the algorithm 

The proposed algorithm depends on three stages: 

3.2.1 Initialization 

Let's note by Y the input signal; we have a library that contains Nm w wavelets. To every 

wavelet ^|we associate a vector whose components are the values of this wavelet 

according to the examples of the training sequence. We constitute a matrix thus constituted 
Vw of the blocks of the vectors representing the wavelets of every mother wavelet where the 
expression is: 



A New Algorithm for Initialization and Training of Beta Multi-Library Wavelets Neural Network 



203 



KM 

KM) 



v x N M 
KM 



KM 



KM 

KM 



K M M 

V»(x 2 ) 



V"M 

v N M M 



#(%,) 



(9) 



(10) 



V;(x N ) ... V' N {x N ) ... V/(x N ) ... Vi{x N ) ... V i M {x N ) 

VJt,d) ={V/}i = [l..N]J = [l..M] 

We note by: 

• g(x) the constructed network, 

• Nw=lthe number of wavelet, 

• T={ti } i=[l..N] the translations, 

• D={di } i=[l..N] the dilations. 



3.2.2 Selection 

The library being constructed, a selection method is applied in order to determine the most 
meaningful wavelet for modeling the considered signal. Generally, the wavelets in W are 
not all meaningful to estimate the signal. Let's suppose that we want to construct a wavelets 
network g(x) with m wavelets, the problem is to select m wavelets from W. 
To the first iteration, the signal is Y = Yi, and the regressors vectors are the V w (t,d) defined 
by (10). The selected regressor is the one for which the absolute value of the cosine with the 
signal Yl is maximal. The most pertinent vector from the family Vi carries the index i per ti 
that can be written as the following manner: 



iperti (i J) =ar § max 



y W: 



\Yl\\K J 



withi=[l..N],j=[l..M] 



(11) 




Fig. 2. Selection of the pertinent vector 
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Once the regressor V. 1 is selected, it can be considered like a parametrable temporal 
function used for modeling Y. We calculate the weight Wi defined by: 



K 1 



(12) 



We define the normalized mean square error of training (NMSET) as: 

NMSET(co iper , , t iperl , d iperl ) = 1 f (Y(k) - co iperl * V t f 

N k=l 



(13) 



With Y(k) is the desired output corresponding to the example k, and co *V is the 

iP er * l pert 

wavelets network output corresponding to the example k. 

3.2.3 Optimization of the regressor 

The optimization of the regressor is made by using the gradient method. Let's note by: 

e(x) = Y d (x) - Y(x) , with Yd : desired output and Y : network output. 



dNMSET S , xvr , 

., 1 i=i 



dw. 



ipert *-l 



V " ipert l j 



dNMSET S , , 
= > e(x)w 1- 



a^ 



^ a ipert *■ J 

dd ipe J 



dNMSET ^ . . 

-— = Z u e(x)w ip l- 

„ 1 7=1 



d¥ 



5t„ 






(14) 



(15) 



(16) 



This optimization has the advantage to be fast because we only optimize here the three 

structural parameters of the network. 

After optimization, the parameters : f pt \ , d opt 1 / co opt 1 of the regressor V 1 are adjusted, 

pert pert pert pert 

and are solutions of the optimization problem defined by: 



VT = V (G) op ' ,t° l ,d opt ) 

» npr t l npr t l npr t l npr t l npr t 



(17) 



Considering the optimal regressor, we reset the network with this regressor that is going to 
replace the old in the library and the orthogonalization will be done using the new 
regressor. 
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V w = 



v;(x 2 ) - f;(x 2 ) -v^( X2 )- vr(x 2 ) 



V?{x N ) Vl(x N ) -V°»( Xn )_ V?(x n ) 

After one iteration we will have: 

v wl(t ,d)={v i M H1 ., ] , M1 .. M] u{^;} 



tptri l p*rt 





Space of NxM dimensions 



V N M (x 2 ) 



V N M (x Ni ) 




(18) 



(19) 



Fig. 3. Regressor optimization 

3.2.4 Orthogonalization 

The vectors V j are always linearly independent and non orthogonal (because N » Mw). 

The vectors V j generate a sub-vector-space of M*N dimensions. We orthogonalize the M*N 
-1 remaining regressor, and the vector Yl according to the adjusted regressor V opt 1 • 



r J ± - J/J 



K J -(K 



Y 1 l = Y-(Y 

Therefore, we make the library updating: 



V opt \)v opt \ 

pert I pert 



V opt \)v opt \ 

pert I pert 



(20) 
(21) 
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V?W •■■ V^(x 2 ) -vr{x 2 ) 



v^( Xl ) ... vrM ... v n m± m 
vi L {x 2 ) ... vr(x 2 ) - c 1 ^) 



(22) 



C(%) c(%,) •■< (%) c(%,) - f, Ml K) - c 1 ^,) 

We will have: 

V,M^ 1 )h 1 4H...*(v, (23) 

Y 1 and j K 7 [ are respectively what remains from the signal and regressors in the 

orthogonal space to V° p 1 . 

The model being at this stage, g(X) = col * V opt \, can be represented by the figure 4. 



V/ V t 1 If' 1 Yl 

■.t»_ 'j«.^ "jJiH --.i 





Space of dimension N 



Fig. 4. Orthogonal projection on the optimal regressor 

To the following iteration we increment the number of N W =N W +1 wavelet. We apply the 
same stages described above. Let's suppose achieved i-1 iterations: We did i-1 selections, 
optimizations, and orthogonalizations in order to get the i-1 adjusted regressors 
( V opt \ , , V opt i-1) we reset i-1 parameters of the network. 

l pert l pert 

The network g(x) can be written at the end of the iteration i-1 as: 



g(x)=K'*Kt 



(24) 



We have Nw-i+1 regressors to represent the signal Yi in a space of N*M-i+l dimensions 
orthogonal to ( V opt I,-, V opt i - 1 ) 

pert pert 

We apply the same principle of selection as previously. The index i per ti of the selected 
regressor can be written as: 
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V\ V i 
W(iJ) = arg max } - I (25) 



W V i 



with (i=[l..N], j=[l..M]) \ { i pert i i per i - 1 } 

Since the regressor is orthogonal to ( V opt \ ,■■•, V opt i — 1 ), we make the updating of the library, 

pert pert 

then we optimize the regressor and finally an orthogonalization. 

Finally, after N iteration, we construct a wavelets network of N wavelets in the hidden layer 

that approximates the signal Y. 

As a consequence, the parameters of the network are: 

T opt = Lop, ) 

<■ "'" > i pen =[l-Nop,] 

d "' = \d° p ' } (27) 






C0° pt =\(oT\ (28) 



The obtained model g(x) can be written under the shape: 

N opt 

g(x) = ]£G>?**V«" (29) 

i=\ 

4. Interpolation of 1D data 

4.1 Mathematical formulation 

The mathematical formulation of the interpolation of ID data can be presented in the 
following way: 

Let the set of the points E = Mx k ,y k )/k = 0, 1, ...., k\ . We want to recover N samples of 

i(x)asf(x) = y k fork = 9 l 9 -,k. 

The set E represents the constraints of the problem. With this formulation the function f (x) 
pass inevitably by the set points of E. In practice, the constraints can contain noise. In this 
case, the signal that we want to rebuild doesn't necessarily pass by the points of the set E, 
the interpolation becomes then a problem of approximation: 

Once we know the function f(x) on the set of the domain x e[0,...,N], the problem is to 
recover f(x) for x> N. This formulation will be called extrapolation of the signal f(x). 
Interpolation is a problem of signal reconstruction from samples is a badly posed problem 
by the fact that infinity of solutions passing by a set of points (Figure 5). For this reason 
supplementary constraints that we will see in the presentation of the different methods of 
interpolation, must be taken in consideration to get a unique solution. 
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Fig. 5. Infinity of curves passing by a set of points 

4.2 Interpolation of 1D data using wavelets networks 
4.2.1 Regularization 

The variationnel method is often used for the regularization [Buli80], [Boor63], [Terz86]. The 
principle of this method is to minimize the function ^(f) defined as the following manner: 



%(f) = aS(f) + yC(f) with a=l-y 



(30) 



The function ^(f) is the sum of a stabilizing function S(f) and of a cost function C(f). The 
parameter ye [0, 1] is a constant of adjustment between these two functions. When y goes 
toward zero, the problem of interpolation turns into a problem of approximation. The 
stabilizing function S(f) fixes the constraint of curve smoothing and it is defined as the 
following way: 



s(f)=j y- d, 



(31) 



Where, D represents the domain of interest. The cost function C(f) characterizes the anomalies 
between the rebuilt curve and the initial constraints, this function can be written by: 



C(j)= ^[A^-y.Y 



(32) 



Where Ep = {(xk,yk) | k = 1,2,... ,K} represents the set of the known points or the signal 
constraints. 



4.2.2 Discretization 

With the regularization, it is difficult to get an analytic solution. The discretization is useful 
and several methods can be used. Grimson [Grim83] uses the finished differences to 
approximate the differential operators; while Terzopoulos [Terz86] uses finished elements to 
get and solve an equation system (Grimson and Terzopouloses used the quoted methods 
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while treating the case 2D). In the approach that follows, the function f is written like a 
linear combination of basic functions: 



/(*) =JW, % (x) 



(33) 



Where N is the domain dimension and Wi the coefficients. The basic functions ^(x) are 

localized to x = iA, ^(x) = x ¥ (x- iA) 

While substituting (33) in (32) and (31), the function ^(f) can be rewritten as the following 

way: 



^=a££w, tyWj + T X £w, ¥,(x k )-y k 

i=0 j=0 1 



Where tij is a function of basic functions: 



-i 



d 2X ¥, 



i\ dx2 



d 2X ¥j 

dx 2 



dx 



(34) 



(35) 



Several wavelets functions can be used like activation function. The Figure 6 gives the curve 
of a new wavelets based on Beta function given by the following definition: 

4.2.3 Definition 

The ID Beta function as presented in [Alim03] and in [Aoui02] is a par ametr able function 

defined by pyx) = J3 x (x) with xo, xi, p and q as real parameters verifying: xo <xi, and: 



x = 



px x + qx 



(36) 



p + q 

Only the case for p>0 and q>0 will be considered. In this case the Beta function is defined as: 



fi(x)- 






else 



(37) 



4.2.4 Axiom [Ben Amar 2006] 

V n e IN, (p,q)edi 2 , p = q, and n < p; the n th derivatives of ID Beta function are wavelets 

[Amar06]. Let's note by Beta" the n lh derivative of Beta function. 



x ¥ rl (x) = Beta" 



d n /3{x) 
dx" 



(38) 
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Fig. 6. Different shapes of Beta wavelets 



4.2.5 Example: Interpolation of 1D data using classical wavelets network CWNN 

In this example, we want to rebuild three signals Fi(x), F2(x) and Fs(x) defined by equations 
(39), (40) and (41). We have a uniform distribution with a step of 0.1 known samples. For the 
reconstruction, we used a CWNN composed of 12 wavelets in hidden layer and 300 
trainings iterations. We note that for Beta wavelets we fix the parameter p=q=30. 



F l (x) = 



-2.186x-12.864 for xe[-\0,-2[ 
4. 246x for jce[-2,-0[ 
10exp(-0.05x-0.5)sin(x(0.03x + 0.7))/orxe[0,10[ 



(39) 



F 2 (x) = 0.5x sin(x) + cos 2 (x) for x e [-2.5, 2.5] 



(40) 



F 3 (x) = smc(1.5x)/orxe[-2.5,2.5] (41) 

Table 1. gives the final normalized root mean square error (NRMSE) of test given by 
equation (42) after 300 trainings iterations for the Fi, F2 and F3 signals. 
We define the NRMSE as: 



NRMSE- 






(42) 



IU-) 2 
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Where N is the number of sample and y { the real output. 





M-hat 


Polywog 1 


Slog 1 


Beta 1 


Beta 2 


Beta 3 


Fi 


7.2928e-2 


4.118e-2 


3.1613e~2 


1.7441e-2 


1.2119e-2 


1.4645e-l 


F 2 


7.6748e-8 


1.0343e-6 


2.3954e-6 


3.7043e-8 


5.3452e-9 


1.2576e-9 


F 3 


3.1165e-7 


2.3252e-6 


1.2524e-6 


5.5152e-7 


1.9468e-7 


1.9674e-6 



Table 1. Normalized root mean square error of test for different activation wavelets function 

4.2.5 Example: interpolation of 1D data using MLWNN 

We intend to approximate the Fi, F2, F3 using MLWNN, composed of a library of 6 mother 
wavelets (from Beta 1 to Beta 3 , Mexican hat, polywog 1 and Slog 1 ), in the same condition as 
the example of approximation using CWNN. 





M-hat 


Polywog 1 


Slog 1 


Beta 1 


Beta 2 


Beta 3 


NRMSE 


Fi 


4 





3 


3 





2 


3.79841e-3 


F 2 


1 


3 


3 


1 


1 


3 


4.66143e-ll 


F 3 


1 


3 


1 


1 


1 


5 


3.84606e-8 



Table 2. Normalized root mean square error of test and selected mother wavelets 

To reconstruct the Fi signal with a NRMSE of 3.79841e-3 using 12 wavelets in hidden layer 
the best regressors for MLWNN are: 4 wavelets from the Mexican hat mother wavelet, 
wavelet from the polywog 1 , 3 wavelets from the Slog 1 , 3 wavelets from Beta 1 , wavelet from 
Beta 2 and 2 wavelets from the Beta 3 mother wavelets. When using a CWNN the best 
NRMSE of reconstruction is obtained with Beta 2 mother wavelet and it is equal to 1.2119e-2. 
For F 2 signal the NRMSE is equal to 4.66143e-ll using MLWNN whereas it is of 1.2576e-9 
using CWNN with Beta 3 mother wavelet. Finally for F3 signal we have a NRMSE of 
3.84606e-8 for a MLWNN over 1.9468e-7 as the best value for a CWNN. 

5. 2 Dimensional data interpolation 

Previously, for every described method in ID, the case in two dimensions is analogous, 
while adding one variable in the equations. 



5.1 Mathematical formulation 

The mathematical formulation of 2D data interpolation can be presented in an analogous 
way to the one described in the ID case [Yaou94] (we will suppose that we want to rebuild 
an equally-sided surface): 

Let E the set of the points E- [yx k ,y k ,z k )/ k = 0,1, ....,£} we wants to recover NxN 

samples of f(x, y) as f(xk , yk) = Zk for k = 1,. . ., K. 

The set E represents the constraints of the problem. With this formulation the function f (x, y) 

passes inevitably by the points of the set E. In practice, the constraints can be noisy. In this 
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case, the signal that we want to rebuild doesn't necessarily pass by the points of the set E. 
The interpolation becomes then a problem of approximation. 



5.2 Method using wavelets networks 

The formulations for the 2D case are given by: 



™-Bi£ 



~(d 2 f) 


2 

+ 2 


v dxdy j 


2 
+ 


(d 2 f) 


2 



da 



c(/)=IZ[/(w*)-z*f 

x k eE y k eE 



(43) 
(44) 



f=0 7=0 



(45) 



Since the interpolated basic functions (wavelets) are separable, this will always be the case 
in this survey: 



¥,..(*, y) = ¥,.(x)¥.O0 



'J" JJn 



w. 



s Zv f,, )fd 2x ¥ ^ 






v dx j 



+ 2 



dxdy 






f d 2 v t x 



{ d 2 ^ r ^ 



(46) 

da (47) 



5.2.1 Example: approximation of 2D data using CWNN 

To compare the performances of the method using wavelets networks and classic wavelets 
networks, four surfaces have been chosen. These surfaces are used by Franke [Fran79] and 
are also described in [Renk88] for x and ye [0,1]: 



S l {x 9 y)= 



[(x-0.5) 2 + (y-0.5) 2 } 



x + 2^| ( 2 — x\ ( y + 2\ ( 2 — y^" 



, v 3.2(l.25+cos(5.4y)) 

S 4 (x,y)=(x 2 -y 2 )sin[5x) 
The Figure 7 represents these four surfaces of 21X21 samples. 



(48) 
(49) 
(50) 
(51) 
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Fig. 7. Surfaces Si(x, y), S2(x, y), Ss(x, y) and S4(x, y) represented by their 21X21 samples 
The four following samplings are considered in order to rebuild these surfaces: 
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Fig. 8. Samplings considered for reconstruction of the surfaces Si(x, y), S2(x, y), Ss(x, y) and S4(x, y) 
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Surfaces 


Sampling 


Beta 1 


Beta 2 


Beta 3 


M-hat 


Slog 1 


Polywog 1 


Si 


1 


4.5472e-6 


1.5881e-5 


1.2783e-5 


3.6561e-5 


6.2610e-6 


8.0144e-7 


Si 


2 


3.6612e-6 


1.9917e-6 


2.6849e-6 


4.2004e-5 


1.4369e-5 


7.6421e-6 


Si 


3 


1.7730e-6 


2.3737e-6 


1.5610e-6 


3.5810e-6 


9.3080e-6 


5.2441e-7 


Si 


4 


2.4055e-7 


4.1873e-6 


8.6403e-7 


7.6317e-6 


5.2528e-6 


6.7456e-6 


Surfaces 


Sampling 


Beta 1 


Beta 2 


Beta 3 


M-hat 


Slog 1 


Polywog 1 


s 2 


1 


5.2167e-6 


7.9647e-6 


3.0107e-6 


2.2445e-5 


8.4106e-5 


5.2539e-6 


s 2 


2 


6.2544e-7 


1.2262e-6 


7.9519e-7 


4.3444e-5 


4.5899e-5 


9.1619e-7 


s 2 


3 


7.2351e-7 


2.8619e-6 


1.9735e-6 


8.7916e-5 


3.0003e-5 


4.7101e-7 


s 2 


4 


2.1238e-7 


8.2108e-7 


5.5915e-7 


1.7681e-5 


1.0731e-5 


1.0287e-6 


Surfaces 


Sampling 


Beta 1 


Beta 2 


Beta 3 


M-hat 


Slog 1 


Polywog 1 


S 3 


1 


9.3210e-4 


7.4748e-4 


6.0998e-4 


4.4958e-3 


2.7186e-3 


9.4392e-4 


S3 


2 


4.0176e-4 


2.6073e-4 


1.5930e-4 


1.2234e-3 


1.2249e-3 


7.5690e-4 


S3 


3 


2.3361e-4 


2.7506e-4 


2.7690e-4 


1.1170e-3 


8.5758e-4 


1.3850e-4 


S 3 


4 


1.1620e-4 


2.6754e-4 


1.9940e-4 


2.1405e-4 


1.1916e-4 


2.2120e-4 


Surfaces 


Sampling 


Beta 1 


Beta 2 


Beta 3 


M-hat 


Slog 1 


Polywog 1 


s 4 


1 


3.8187e-3 


4.6391e-3 


2.2186e-3 


3.3708e-2 


3.0934e-3 


4.8993e-3 


s 4 


2 


1.5365e-3 


2.4675e-3 


1.3099e-3 


9.9976e-3 


4.8904e-3 


2.5167e-3 


s 4 


3 


3.7501e-3 


1.7048e-3 


5.4345e-4 


1.2712e-2 


3.0359e-3 


2.2196e-3 


S4 


4 


3.2641e-4 


1.4836e-3 


6.2990e-4 


1.5668e-3 


2.0464e-4 


2.2768e-4 



Table 3. Normalized Root Mean square error of test for the surfaces Si(x, y), S 2 (x, y), Ss(x, y) 
and S 4 (x, y) using CWNN 

Table 3 represents the NRMSE of reconstruction of the four considered surfaces, using 

classical wavelets network constructed with 12 wavelets in hidden layer and based on Beta, 

Mexican Hat, Slog 1 and Polywog 1 wavelets [Belli05]. This table informs that the number of 

samples to consider as well as their disposition for the reconstruction is important. 

For a same number of samples, it is preferable to use a uniform sampling than a non 

uniform one, the more the number of samples is important and the better is the quality of 

reconstruction. 



5.2.1 Example: approximation of 2D data using MLWNN [Bellill07-b] 

The same surfaces are used in the same conditions but using MLWNN with a library 

composed of two mother wavelets (Beta 1 and Beta 3 ). Experimental results are given in the 

following table. 

Table 3 and table 4 inform that the number of samples to consider as well as their 

disposition for the reconstruction is important: 

For a same number of samples, it is preferable to use a uniform sampling than a non 

uniform one, the more the number of samples is important and the better is the quality of 

reconstruction. 
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When comparing table 3 and table 4 we can say that the performances obtained in term of 
NRMSE using the MLWNN algorithm are often very better that the one obtained with the 
CWNN. This shows that the proposed procedure brings effectively a better capacity of 
approximation using the parametrable Beta wavelets [Belli07-b] . 
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Table 4. Normalized Root Mean square error of test for the surfaces Si(x, y), S 2 (x, y), S 3 (x, y) 
and S 4 (x, y) using MLWNN 

6. 3 Dimensional data interpolation 

The case of 3 dimensions is analogous to ID or 2D case. The reconstruction of sampled data 
using wavelets networks is deduced from the ID or 2D case. 



6.1 Mathematical formulation 

The mathematical formulation of 3D data interpolation can be presented in an analogous 
way to the one described for the ID case (we will suppose that we want to rebuild an 
equally-sided volume): 

Let E the set of the points E = {(x k ,y k ,z k ,q k )/ k = 0, 1, , A:} we wants to recover NxNxN 

samples of f(x, y, z) as f(xk , yk, Zk) = qk for k = 1,. . ., K. 

The set E represents the constraints of the problem. With this formulation the function 

f (x, y, z) passes inevitably by the points of the set E. In practice, the constraints can be noisy. 
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In this case, the signal that we want to rebuild doesn't necessarily pass by the points of the 

set E. The interpolation becomes then a problem of approximation: 

The problem of extrapolation is to recover the values of the function f (x, y, z) for x, y and z 

not belonging to the domain of interpolation. 

The problem of reconstruction of a volume from samples is a badly definite problem 

because an infinity volume passing by a set of points exists. For this reason, some 

supplementary constraints must be taken into consideration to get a unique solution. 

6.2 Method using wavelets networks 

Formulations for the 3D case are given by: 
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Since the interpolated basic functions (wavelets) are separable, this will always be the case 
in this survey: 



V ijr (x,y,z) = V i (x)V j (y)V r (z) 



(55) 
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6.3 Example: approximation of 3D data using MLWNN 

We used the GavabDB 3D face database for automatic facial recognition experiments and 
other possible facial applications like pose correction or register of 3D facial models. The 
database GavabDB contains 427 images of 3D meshes of the facial surface. These meshes 
correspond to 61 different individuals (45 male and 16 female), and 9 three dimensional 
images are provided for each person. The total of the database individuals are Caucasian 
and their age is between 18 and 40 years old. 
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Each image is a mesh of connected 3D points of the facial surface without the texture 

information for the points. The database provides systematic variations in the pose and the 

facial expressions of the individuals. In particular, there are 2 frontal views and 4 images 

with small rotations and without facial expressions and 3 frontal images that present 

different facial expressions. 

The following experiment is performed on the GavabDB 3D face database and its purpose is 

to evaluate the MLWNN that we employ against the CWNN in term of 3D face 

reconstruction. 

For faces reconstruction quality measurement we adopt the common use of NMSE given by: 



NRMSE = 



1 



TV *M 



»• = ! 7 = 1 

N M n 



(57) 



Where Z. . is the depth value of pixels in the reconstructed face; and Zy is the depth value 

of pixels in the original face. To check the utility of the MLWNN, experimental studies are 
carried out on the GavabDB 3D face database. We used 61 frontal views with two different 
dimensions: 20*20 and 50*50. The results of comparison are presented in Table 5. 
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Table 5. Evaluation in term of NMSE of 3D face reconstruction using MLWNN and CWNN 
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7. Conclusion 

In this chapter, we described a new training algorithm for multi library wavelets network. 

We needed a selection procedure, a cost function and an algorithm of minimization for the 

evaluation. To succeed a good training, we showed that it was necessary to unite good 

ingredients. Indeed, a good algorithm of minimization finds a minimum quickly; but this 

one is not necessarily satisfactory. 

The use of a selection algorithm is fundamental. Indeed, the good choice of regressors 

guarantees a more regular shape of the cost function; the global minima correspond well to 

the "true" values of the parameters, and avoid the local minimum multiplication. So the cost 

function present less local minima and the algorithms of evaluation find the global 

minimum more easily. 

For the validation of this algorithm we have presented a comparison between the CWNN 

and MLWNN algorithm in the domain of ID, 2D and 3D function approximation. 

Many examples permitted to compare the capacity of approximation of MLWNN and 

CWNN. We deduce from these examples that: 

• The choice of the reconstruction method essentially depends on the type of data that we 
treat, 

• The quality of reconstruction depends a lot on the number of samples used and on their 
localizations. 

Also we have define a new Beta wavelets family that some one can see that they are more 
superior then the classic one in term of approximation and we demonstrate in [BELLIL07] 
that they have the capacity of universal approximation. 

As future work we propose a hybrid algorithm, based on MLWNN and genetic algorithm 
and the GCV (Generalised Cross validation) procedure to fix the optimum number of 
wavelets in hidden layer of the network, in order to model and synthesis PID controller for 
non linear dynamic systems. 
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1. Introduction 

Robotic platforms can be used in countless applications and in the most varied branches of 
activities. Such results, presented over the last two decades, can be verified in Armada et al., 
2003 and Virk, 2005. 

It deals specifically with robots provided with legs and with the capability, or ability, to 
climb vertical surfaces, while many other applications/ solutions can be found. As an 
examples: robots to climb lower parts of bridges (Abderrahim et al., 1999), to crawl inside 
pipes for inspection purposes (Galvez et al., 2001), implemented to perform solder 
inspection works in nuclear plants (White et al., 1998), to climb metallic structures for 
inspection purposes (Armada et al., 1990). More examples can be found in marine industry 
applications, such as walking robots to check internal parts solders in ship hulls, climbing 
robots for parts solders (Santos et al., 2000) and (Armada et al., 2005), climbing robots for 
paint cleaning, underwater robots for ballast tank inspection, and underwater robots for hull 
cleaning (Santos et al, 1997a, 1997b). 

Robots were demonstrated to be the ideal option for many such applications due to the fact 
that the working environment is difficult to access or even hazardous or risky for human 
beings, such as exposure to hazardous substances or environments and risk conditions. 
Productivity increase and quality issues are also extremely relevant and are considered. 
However, besides the varied applications and areas mentioned above, there still remains a 
little explored area: environmental research. As in any other area, different applications or 
problems can be addressed or solved with the help of a robotics platform. As an example, 
one can mention the tasks: 

• Gathering of Botanical Specimens: gathering flower and plant specimen is 
fundamental for biodiversity studies. Several host species are found in high trees, and 
their collection is actually risky. Thus, this is an application where a robotics platform 
with tree climbing capability can be used to minimize risks for the researchers involved 
in this type of activity. 

• Gathering of vegetable material: collection of vegetable material is a fundamental 
activity for phytochemistry. Every pharmaceutical preparation that uses as raw material 
plant parts such as leaves, stems, roots, flowers, and seeds, with known 
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pharmacological effect is considered phytotherapic (extracts, dyes, ointments and 
capsules). Thus, just as in botanical collection, this is an activity that can be 
accomplished by a robot operating in large-sized trees, therefore minimizing risk for the 
humans involved in the collection. 

• Gathering of Insect Specimens: usually, for collecting insects in higher levels, nets 
spread around the tree and a source of smoke below it are used. The great discussion 
about this technique is that one not only captures the desired insects, but one ends up 
killing most of the insects that inhabit that tree as well. Thus, one proposes to use a trap, 
positioned in the robot, containing a pheromone or equivalent substance as bait, 
specific for the insect species one wants to capture. One can even adapt cameras and 
sensors in the trap to gather images of the moment of capture. With a trap thus 
implemented, one reduces the negative environmental impact of the capture. Another 
relevant issue would be the possibility that the robot moves along the day to capture 
varied insect specimens at different heights in different hours, to check for variations in 
insect populations according to the height, and the hour of the day. 

• Climatic studies: climatic or microclimatic studies refer to works on 
microenvironments in native vegetation and/ or reforested areas. It is important to 
study the different energy flows: horizontal and vertical. The vertical directly reflects 
the results of solar radiation, which decisively influences the horizontal energy flows: 
air masses, hot and cold fronts, action centers. Solar radiation determines the whole 
system, and may be analyzed according to its elements: temperature, pressure, and 
humidity, greatly influencing biogeographic characteristics, geomorphologic and 
hydrologic phenomena etc. Thus, the robot can be equipped with sensors for such 
measures, to collect data on the desired elements. 

• Studies on biosphere/atmosphere interaction: biosphere environments are the group of 
biotic or abiotic factors that interfere in the life conditions on a certain region of the 
biosphere. In the case of forests the aerial environment is studied, and the most important 
elements to consider are: light, oxygen, ice formation, winds, humidity and carbon gas. In 
order to register all this information, the robot can be endowed with specific sensors for 
each kind of required measure, to collect data regarding the elements at hand, and to 
provide information on them regarding both height and time variations. 

• Studies on arboreal fauna: fauna studies are hampered by the existence of many 
leaves, or very dense treetops, as the lack of existing natural light hinders the 
observation of the species. Other usually relevant points are the difficulty to obtain a 
proper angle due to the great heights involved, and the very presence of human beings 
in that specific environment, easily detected by their movements, noise and odors. For 
this type of task, the robot can be fitted with cameras to capture both static and dynamic 
images. These can then be stored locally in some type of memory card, or transmitted 
via communication interface to a base station. 

• Sensors Network: robots carrying a group of sensors and fitted with a communication 
interface, for instance Wi-fi or other similar technology can be dispersed in the forest to 
capture data regarding the ecological behavior in the area at hand. Measurements such 
as the ones already mentioned in climatic studies and biosphere/ atmosphere 
interaction can be shared among robots or even retransmitted among robots to reach 
the base station, without the need for the researcher to "pay visits" to the reading 
points. 
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2. Kamanbare platform 

Purporting the main goal to climb trees for environmental research applications, to be 

applied in tasks such as presented above, a bioinspired robotic platform named Kamanbare 

was proposed. 

The project's main application is climbing trees for non-invasive search purposes, reaching 

points (at high altitudes) that may offer risk to humans. 

The development was driven mainly to seek for robot stability and efficiency regarding the 

paws. The adopted line of research is an implementation inspired in nature. One must stress 

here that the system doesn't mimic nature, by copying an animal or insect to accomplish the 

desired work, but rather, a robot development project that combines the climbers' best 

characteristics (insects and lizards) considering the available technologies and the associated 

cost/ benefit ratio, in other words, some parts were inspired in certain solutions found in 

nature, while other parts were inspired in different elements. 

Ensemble stability, contact stability (paws), contact position, and contact force (pressure) 

were defined and used to implement the strategies and techniques for dynamic and static 

control of the system. 

3. Chameleons: the biological inspiration 

Differently from any other reptile or lizard, chameleons have paws that were created, or 
adapted, to adequately clasp the different types of branches or shrubs existing in its 
environment. 

The chameleon paws evolved to provide them with the best possible maneuvering and grip 
capabilities on trees. The paws are actually forked, with three fingers on one side and two 
on the other. Frequently they also have powerful and sharp nails (or claws) that can grip the 
surface which they hold on to. This unique arrangement allows them to position their paws 
completely around the branches or shrubs on which they move around, giving them an 
amazingly strong clasping capability. 

Chameleon paws have five fingers each, divided in two groups (internal and external), one 
side composed of three fingers while the other side has two, as seen in Fig 1. Front paws 
have two fingers pointing to the external side, and three towards the inner side, while rear 
paws are configured in an opposite arrangement, i.e., two inside fingers and three outside 
fingers. This provides the chameleon with the same number of fingers on each side of the 
branch, considering all the paws, allowing a balanced and stable clasping. 




Fig. 1. Details of the chameleon paw, presenting the bifurcation and configuration of the 
fingers 
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4. Mechanical model 

The mechanical structure of the Kamanbare platform consists of a central rigid body with 
four legs, identical and symmetrically distributed. Each leg comprises three links connected 
by two rotating joints, and it is connected to the central body by a third rotating joint, while 
each joint has 1 DOF (degrees of freedom). Identical motor and reduction groups make the 
rotary movements. Fig. 2 shows the kinematic configuration of a leg. Each leg also has a 
paw, which is forked just as the chameleons, but with only two fingers, one on each side, for 
simplification purposes. The paw connects to the leg by a rotating joint, and also has another 
motor and reduction group that provides for its opening and closing movements. 
As each leg has four joints, this means 4 DOF. Considering the four legs, the platform will 
have a total of 16 DOF. Therefore, sixteen motor and reduction groups were necessary to 
produce the global movements. 




Fig. 2. Kinematic configuration of a leg 

The main dimensions of the platform are: length of 250 mm, and width varying between 260 
and 740 mm, depending on the positioning of the legs. Its height also depends on the 
posture and positioning of the legs, and can vary between 190 and 310 mm. 
This configuration, with all the parts implemented using two different materials, aluminum 
and polyacetal, comprises an approximate total weight of 1.3 kg, not including the batteries. 
The geometric model obtained can be seen in Fig. 3. This model was used to generate the 
basic locomotion behavior. 
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Fig. 3. Mechanical structure of the Kamanbare platform 



5. Control software architecture 

An architecture was implemented for local control of the Kamanbare platform. This 
architecture corresponds to the robot's functional organization. 

Based on the hardware architecture to be presented in the following section, the 
development of the following systems was accomplished according to Fig. 4. This model is 
based on the architecture implemented for the MARIUS robot (Pascoal et al., 1997) and has 
the following main components described below. 

• Support system: this system controls energy distribution to the platform's electronic 
and electromechanic hardware, and monitors energy consumption as well. This system 
is also responsible for the startup of other subsystems and, during operation, for 
detecting hardware failures, and for starting and controlling the emergency modes. 

• Actuators control system: this system is responsible for controlling the motors, and also 
for controlling the movements of the legs. Information on legs positioning is received 
from the general control system. Data regarding joint positions, as well as the values of 
the electric currents involved, are sent to the general control system. 

• General control system: this system receives trajectory reference information from the 
mission control system. It controls all the robot's movements, sending the necessary 
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commands to the actuators control system. Problems occurring in the path, such as 
obstacles and absence of support points for the paws, are handled in this system. 
Mission control system: this system is the main module, the highest hierarchical level 
of the platform. It is responsible for receiving commands via the communications 
system, and for distributing them to the systems involved. It also stores information on 
the general status of the platform (battery voltage, position of the legs, angles etc.) 
keeping them available in case the Base Station (described in the following topics) 
requests them. This system gathers information from the Environmental inspection 
system to be subsequently forwarded to the Base Station. 

Communication system: this system is the module responsible for the communication 
interfaces existing in the platform, managing communications via Wi-fi and Bluetooth, 
and exchanging data with the Mission control system. 

Environmental inspection system: this system is responsible for gathering data from 
the installed sensors, and for controlling any additional hardware necessary for that 
purpose as well. Every data acquired are sent to the Mission control system. 
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Fig. 4. Kamanbare's Control Software Architecture 



6. Electronic architecture 

Considering the implementation of control algorithms, processing information from sensors, 

the necessary calculations for control and locomotion strategies, interfacing and 

communication with the base station, added to the need of real time control with position 

and force feedback, the involvement of high computing complexity was ascertained, thus 

requiring a processor of advanced architecture. 

Eventually, it was selected then a development kit containing a processor based on the 

ARM9 core, and the deployment of a Linux operating system. 

Other motor control boards were also developed using a Texas Instruments microcontroller 

of the MSP430 family and specific integrated circuits to implement the power actuation 

section, based on the so-called H-bridge technique. 

To implement the control systems for the Kamanbare platform, an electronic architecture 

was defined. Initially considering only one joint, it can be represented in Fig. 5, where the 

main components are seen: a DC motor, a potentiometer and a micro switch. 
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Fig. 5. Representation of a joint 

Thus, for control purposes, the need of a PWM output (motor control), an analog input 

(potentiometer reading, indicating the joint angle), and a digital input (reading the end, or 

beginning, of the joint course) was ascertained. 

As already mentioned, the platform has 16 DOF, corresponding to the need of sixteen copies 

of the joint control system described above. 

As the robot has four legs, one opted for distributing the control individually to each one of 

them. Thus, each leg control module needs four groups as mentioned, namely, three for the 

joints, and one for controlling the opening and closing of the claw. 

One then developed a motor control board for this specific purpose, Fig. 6, based on the 

MSP430F149 Texas Instruments microcontroller, and the L298 integrated circuit (H-bridge). 
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Fig. 6. Motor control board diagram 

Due to the control complexity existing in a robotics platform, it was necessary to adopt a 
main board where the highest hierarchical level control activities were executed. 
As a solution for the main board, the model selected was the TS-7250 by Technologic Systems. 
It was selected because it's compact, contains different standard interfaces, and is based on the 
EP9302 Cirrus Logic processor, with an ARM9 core, Fig. 7. The EP9302 implements an 
advanced processor core: 200 MHz ARM920T with support for a memory management unit 
(MMU). This ensemble allows the use of a high-level operating system, in this case Linux. The 
ARM920T core has a 32-bit architecture with a 5-stage pipeline, offering high performance and 
low energy consumption levels. With a 200 MHz clock, the TS-7250 module offers a 
performance approximately twice as fast as other boards based on 586-core processors. 
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Fig. 7. Main board diagram 
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Thus, the general electronic architecture for the Kamanbare platform was deployed 
according to the diagram in Fig. 8. 
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Fig. 8. Electronic architecture of the Kamanbare platform 



7. Simulink models 

Simulink® and SimMechanicsTM software use a block diagram approach to model control 
systems around mechanical devices and simulate their dynamics. With CAD (Computer- 
aided design) translation, it was possible to combine the power of CAD and SimMechanics 
software. The translator transforms geometric CAD assemblies into Simulink blocks. 
The CAD model of the Kamanbare robotic platform, described in the section 2 - Mechanical 
Model, was designed using Soliworks. 

Thus a Kamanbare model was generated using the SolidWorks-to-SimMechanics translator. 
This translation process is based on two major steps: to export the CAD assembly into a 
physical modeling XML format file and to import the generated XML file into a Sim 
Mechanics model in Simulink. 

For the translation procedure, some configurations like the name of the joints and legs were 
adopted as well the desired movement direction and the gravitational acceleration direction 
of influence, as demonstrated in Fig. 9. 

The resultant model obtained for the Kamanbare platform is depicted in the next in Fig. 10. 
For a better understanding and visualization the legs were represented as model blocks. 
Due the fact that all four legs are identical, one detailed leg model block, the Front Left Leg, 
is described in Fig. 11. 

A signal builder was also implemented with the main function of gait generation, here 
providing the correct angle references for all joints as function of time, Fig. 12. 
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8. Gait generation 

Gait generation is the formulation and selection of a sequence of coordinated leg and body 

motions that propel the robot along a desired path. 

Free gaits are gaits in which any leg is permitted to move at any time. Fixed, or regular, gaits 

are those in which a specific pattern of leg movement is imposed. All animals locomote with 

fixed gaits. A tenet of behavioral systems is that the emulation of animal behavior, hence 

fixed gaits, is sufficient. 

Up to now, just a basic gait was implemented based on the biological inspiration cited 

before: the chameleon, Fig. 13. 




Fig. 13. Biological inspiration for the gait implemented for initial simulations 

The gait shown in Fig. 14 represents the variations of the joint angles as function of the time 

for all the four legs for a movement on a vertical and plane surface. 

For a better understanding while analyzing the Fig. 14 a visit on Fig. 9 will help the 

identification of the names adopted here. Just for clarification, the configuration adopted 

follows: 

• Front Left Leg: Leg 1 with the joints Jll, J12, J13 and J14; 
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• Rear Left Leg: Leg 2 with the joints J21, J22, J23 and J24; 

• Front Right Leg: Leg 3 with the joints J31, J32, J33 and J34; 

• Rear Right Leg: Leg 4 with the joints J41, J42, J43 and J44. 

As explained before, this is a basic gait and it was generated manually observing the 

chameleon model of movement. 

No kind of gait optimization was implemented. The idea here is to have a basic gait to start 

simulations in order to obtain some results like mechanical interferences between legs while 

moving and necessary torques for the desired movement. 

Gait optimizations in order to obtain a smooth movement as well as the better use of the 

energy, or energy saving, while moving the joints will be studied and presented on future 

works. 
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Fig. 14. Gait diagrams corresponding as joint angles positions 
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9. Simulation results 

Regarding the movement generation, or gait, only the simplest way was initially 
implemented, i.e., the robot follows a straight path at the highest speed allowed by the 
surface (no waiting time will be introduced, besides those produced by the legs when 
searching for support points). 

The proposed realistic and practical SimMechanics model is further employed to simulate the 
designed controller which is implemented using Simulink, in this case just a gait generator. 
As first result obtained are the models machines presented in Fig. 15 and Fig. 16. These 
models are quite interesting and useful while running simulations for the reason that all 
joints and legs movements can be observed. 



^ 




Fig. 15. Kamanbare's model machine in convex hulls view mode 




Fig. 16. Kamanbare's model machine in convex hulls + ellipsoids view mode 
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As explained before, both the body movement and the foot trajectories are given, as well the 
joint angles of the legs, as a gait. Using these inputs and running simulations in an inverse 
dynamics option, all the joint torques can be determined. 



time (s) 
Fig. 17. Computed torques for the group of joints 1 

Joint J12 





Fig. 18. Computed torques for the group of joints 2 
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The results presented in the following figures are the necessary torque to produce the 
desired movement, or gait, presented in Fig. 14. The torques are grouped by the function, i.e, 
torques for the joints with the same functionality, or the same mechanical position and 
configuration, for legs 1, 2, 3 and 4. 
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Fig. 19. Computed torques for the group of joints 3 
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10. Conclusion 

This work presented the Kamanbare robot, which is a four-legged bionspired platform with 

the main purpose of climbing trees, for environmental research applications. The 

mechanical and electronics structure, and the software architecture, were presented as well a 

Simulink mechanical model and simulations results. 

Based on its special design, this platform offers the possibility of investigating reptile-like 

walking and climbing. 

As a control approach, just a gait generator based on the chameleon mode of movement was 

presented, but it was demonstrated that is sufficient to obtain results on legs movement 

interference and realistic torque estimates. 
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1. Introduction 

On the last decade the development of computers and ICT technology has allowed the 
emergence of a number of new disciplines that have significantly changed daily life and 
Virtual Reality is certainly one of the most interesting novelty. This technology has focused 
much of the work researchers. In fact in recent years virtual reality is used in many areas: 
robotic, simulations, training, entertainment (Video games offers fantastic characters to play 
and a vast interactive worlds to explore), medicine, etc. Even entire training programs can 
take place in a virtual reality. In this scenario at first was proposed various languages to 
introduce the third dimension but not to interact with objects. For against, 3D objects 
interaction can offer extremely realistic representations that enable users to move and react 
in a computer simulated environment. 

To this purpose few attempts have been made to offer the opportunity to build virtual 
worlds in what you are completely immersed and where is possible to interact both objects 
it contains both with people represented by avatar, allowing to hear sounds, see pictures 
and movies, providing perceptions very similar to real ones. 

Though Virtual Reality has reached a mainstream status, actually creating 3D application 
still requires much effort and only with proper, dedicated tools is it possible to produce 
convincing result in a timely manner. 

Therefore in this chapter we introduce Quest 3D, that is a perfect software package for 
creating interactive 3D scene including product presentation, architectural visualizations, 
virtual trainings and computer games. 

Quest 3D is a software for creating interactive 3D scene, in a unique style of programming. 
Instead of having to write thousands of lines of complex code, developers make use of a 
large set of a powerful building blocks. Both flexible and easy to use, Quest 3D appeals to 
designers, programmers and artist alike. 

Working with Quest 3D means developing in a real-time: you are working directly on the 
end result. No time will be lost on compiling code or rendering image. While the logic tree 
scanning goes on, system shows the visualization of scene in the window application. 
Quest 3D has a stunning set of graphics features. Large numbers of animated people, 
vegetation, shadows, fire and smoke effects and realistic water can all be easily added to a 
scene. Advanced features include physics simulation, path finding routines, database 
connectivity and networking support. By this way is possible design very simple scene like 
extremely complex projects: software product always will appear readable and smart. The 
high number of building blocks can be logically regrouped in folder, characterized by 
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different name and colour for the various functions: a structure very similar to Java or C 

class. Blocks take input from the son blocks, which provide services to father. Every 

scanning of blocks follow a depth first manner from left to right in a single frame, realizing 

the real time rendering. 

Quest3D gives the possibility to manage the logic of scene: the behaviour of objects, their 

interactions. 

Programmer can follow an advanced and sophisticated approach of dynamic loading of the 

various portions of program, to isolate the different logical parts. This solution provide the 

possibility to use the all potentialities of graphic card, still limited for household approach of 

so heavy software programs. 



2. User Interface 

Quest3D provides an user-frienly approach, by a graphic user interface, easy understandable, 

formed by windows, menu and other direct interaction elements with the developer. Let's 

show the elements: 

This is more productive than worrying about how to achieve consistent results in different 

hardware and software environments. 

Toolbar for working on a scene through the different perspectives 
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Channel sections show the structure of building blocks of a scene 
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Animation section allow to animate a scene, deciding the behaviour in different temporal 
instants 




In Object section is possible select single objects which form the scene and its surfaces 
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Path/Nature section allow to add naturalistic elements and phenomenon which make more 
relistic the scene 
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The elements which compose the scene are not created suddenly, but belong to the available 
channel set 
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To semplify programing, Quest3D provides every kind of templates: predefined objects, 
cameras, lights, instruments to add audio file o creating animations, logical-mathematical 
funtionalities 
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3. Channels 

Quest3D programs are made with building blocks. These building blocks are called channels. 
Each channel has a specific function. 




242 



Advances in Robotics, Automation and Control 



The small black squares above and beneath a channel are called link squares. Channels can be 
connected to each other by lines between top and bottom link squares. 




In this image, the upper channel is called the Parent. The lower channel is called the Child. 

Children are often used as input or output data for Parents. 

The small black squares show which necessarily have to be the channel tipology of childreen, 

which parents may use like input. 

Informations about a building block are visible, throught mouse moving, on specific pop-up 

windows. 




In addition to a name, every channel has a Basetype and a Type. A Value channel is of 
Basetype Value but also of Type Value. 
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The connected channels structures compose a Channel Group. A Quest3D project is formed by 
one or more Channel Group. 




Double clicking on a channel to visualize Channel Properties windows, where are specified 
the characteristics of a block (name, value, behaviours). 
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Shortcuts are references to a channel: instead of duplicating a block, is sufficient creating a 
shortcut (throught the menu by right click of mouse) to inheriting all the characteristics of 
the block. 

Shortcuts are references to the original channel. Their only purpose is to keep Channel 
Groups more readable. 

4. Program flow 

A rendering engine is a software which, once download contents (from Internet, like a page 
HTML or XML, an image, ecc.) format the relative informations and show them on screen. 
Quest3D works in real-time. This means that continually executes a complete project and 
upload the preview. 




A complete cycle throught the channel structure is called frame: the set of all frames produces 
the rendering. 
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Framerate defines how many times on one second a program is executed, it depends on 
project complexity and on available hardware. 

5. Modeling import 

Quest3D is not used to create complex models from beginning, it allow to create interactive 
3D scenes architectural presentations, virtual training, computer games, importing entities 
from available modelling tools. 

Models are meanly created by Autocad, 3D Studio Max and Maya, and exported to 
Quest3D. Exporter of different tools are available on www.quest3d.com. 



j 



■ege 



<r^\%%%^ ^Tfp-OMi^O ni^^ - ajij* $i«?.4t#«fli8fcr~ - ^ifli#;i^iB§i8i^.^"^ I | 




— ■ 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 ujj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 ijj 1 1 1 1 1 1 1 \[\ 1 1 1 1 1 1 1 ii u 1 1 1 1 1 1 1 M 1 1 1 1 1 1 1 1 M 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ii j 



W 2fl 3P 



[* ® * s® a m % 


| Standard Primitives j»_ 


|- Object Type | 


r Ant 

Box Core 


Sphere GeoSphere 


Cylinder | Tube | 


Torus Pyramid 


Teapot | Plane | 




|- Name and Color | 




1 ■ 



j None Selected 



. ..... 



Z:| Grid = 10.0m Auto Key Selected ^ h« -*ll E l* »*H <\ <!p ]3T_ rf 



| ! | Add Time Tag. . ^ I Set Key | J* Key Filters... | MM |fo ;] @> |gj f) £. ^ 



6. 3D Objects 

3D scenes consist of virtual objects. Their appearing is defined by a number of elements. 
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A vertex is a single point in 3D space. Vertexes are defined by X, Y, Z coordinates. 




Simple models like a box consist of only few vertexes, while more complex objects can be 
built out thousands of points! 




Surfaces consist of polygons, everyone of them could have an own vertexes number, 
although real-time graphics usually renders 3 or 4 (3 vertexes for each surface of Quest3D 
polygons). 




Individual polygons can be assigned a color, but often, another means to add detail is 
chosen. A 'texture' is an image that can be wrapped around a 3D shape. 
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Textures are wrapped around an object, to define its color and transparency. 




Tipical components of a 3D Object: 

• The 3D ObjectData channel contains the vertices and polygons of an object. It also 
contains information on how to wrap a texture around the shape. 

• The Material channel is subdivided into a number of aspects: Diffuse, Emissive, 
Specular, Power, Alpha Diffuse and Texture Blend Factor. 

• UTexture channel represents the actual image applied on a surface: it contains a 
diffusive colour image (RGB), and an alpha image. 

• II channel Motion, throught its parameters, manages object's position in space. 
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• The Diffuse vector contains an object's colour, and consists of three components: red, 
green and blue, or 'RGB'. 

• The Emissive vector describes the amount of self-lighting the model has. Without lights 
in your scene, an object with an Emissive vector of (0, 0, 0) is black. 

• Lo Specular vector defines the colour and intensity of the highlight, while the Power 
value controls the size of the highlight. 

• The Alpha Diffuse value defines the transparency of an object. 

• The Texture Blend Factor value defines how much of a texture is blended with the 
Diffuse colour. 

7. Animations 

One of the most powerful features of Quest3D is that you can animate virtually every aspect 
of your scene. For instance, 3D objects can be moved, rotated and scaled over time. In 
addition, all of their surface properties can be animated, such as their color and 
transparency values. 3D character models can be brought to life by animating virtual 
skeletons inside of them. Music and sound can be triggered on certain location- or time- 
based events. Menu windows can be moved or faded in and out of the screen. 
To realize animations Quest3D uses: 

• Envelope 

In Quest3D, animation often means changing values over time. The regular Value channel 
can only store one value. To store animation data, the Envelope channel is used. The interface 
of the Envelope channel looks like a mathematical graph. It works in a similar way as well. 
The horizontal axis represents the input (often time), which can be fed to the Envelope by 
connecting a (Timer) Value to its child link. The vertical axis represents the output. Through 
(X, Y) pairs and interpolation, any input can be converted to an output value. 
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• Timers 

The Timer Value and Timer Command channels allow for easy animation control. 

Examples of Timer Commands are 'Play', 'Stop', 'Rewind' and 'Play & Loop'. 



Modeling Virtual Reality Web Application 



249 




Timer Value channels are automatically adjusted for computer speed. In Quest3D, 25 frames 
equal exactly 1 second. 



Timer Value 
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A Timer can be renamed throught the channel property window. 
Each element of the scene can need of an own timer. 



8. Surface properties 

3D object appearing is defined by surfaces properties. 




A 'texture' is an image that can be wrapped around a 3D shape. In real-time graphics, for 
performance reasons, their dimensions in pixels (width and height) must be a power of 2. In 
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Quest3D, they must also be greater than 8. Dimensions of 32x32, 128x256 and 256x256 pixels 

are all valid sizes. 

Compression: 

In addition, textures in Quest3D may be compressed. The image quality might slightly 

decrease, but compressed textures may use up to four times as little memory as 

uncompressed versions. Creating a 3D scene is constantly balancing between quality and 

performance. 




Mipmaps: 

Larger textures on distant surfaces might produce artifacts. By using various versions of 

increasingly smaller size, these distortions may be countered. These smaller versions of the 

same texture are called mipmaps and can be generated automatically by Quest3D. 

UV Mapping: 

Textures are wrapped around a 3D object according to so called 'UV coordinate sets'. UV 

coordinates specify where each pixel of a texture is drawn onto a surface. 

Quest3D can handle simple UV mapping methods such as planar and cubic, shown in the 

image below. These simple UV mappings can be moved and scaled. 




Since UV coordinates are relative, once an object is properly 'UV mapped', textures of any 
size may be applied to the object. 

Wrapping a texture around a 3D object usually requires more control and precision. In 
practice, UV mapping means placing every vertex of a 3D object onto a 2D image. 



♦ A 
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Transparency: 

Transparency of surfaces may be handled in a number of ways, managing the characteristics 

of surfaces in Object section. 

Texture stages: 

More than one texture may be applied on a surface. The maximum number of 'texture 

stages' that can be used per surface depends on the graphics hardware. Most graphics cards 

support two stages, and newer graphics cards support three or four. 

Quest3D tool allows different methods of textures blending on each stages. 




'• fcj% 



; 



v*.*j_y , 







9. Lighting and shadows 

Without contrast between light and dark, it is much harder to establish depth in a scene. 




A right lightings, the shadow contrast are mean elements to create a realistic 3D 
environment! A number of lighting techniques are available in Quest3D: 
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A point light radiates light into every direction. A spotlight is defined by an inner and outer 

cone angle, resulting in a simulated fall-off effect. A directional light is simply a rotation 

vector along which the source emits light. All rays of a directional light are parallel to each 

other. 

A Quest3D light can be colored through its 'Diffuse' vector. The 'Ambient' vector serves as a 

global 'Emissive' vector to all objects affected by the light source. 

A 3D object's illumination may be influenced by changing its Diffuse and Emissive vectors. 

These can also be accessed in the Object Section, on the 'Material' tab. A higher Emissive 

vector will suggest a certain degree of self-illumination or ambient light. 




Shadows for dynamic objects, such as cars or walking characters, can be simulated using 
texture squares. 
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Quest3D also supports real-time shadows. The Stencil Shadow channel can be seen as a 

Render specialised in calculating shadows. The Stencil Shadow channel requires a light source 

position vector to work. 3D Objects may be linked to the StencilShadow by means of a 

SoftwareStencilShadowObject. 

Real-time shadows require a lot of processing power. 




Light mapping is the process of calculating lighting and shadows once, and storing the 
information into a texture. The resulting Tight maps' are then applied on an object on a 
second texture stage, and blended with the diffuse texture. 
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10. Cameras 

A camera defines the point of view from which a scene is presented. Choosing the right 
camera for a project is important, as it influences the way users experience a scene in a very 
direct way. 

Let's show the camera templates available in Quest3D. 
Animation Camera: 

An Animation Camera may be used to guide a user through a scene. The animation camera is 
similar to a real-life freehand cinematic camera. It may also be the preferred camera for non- 
interactive scenes such as visualisations, documentaries or screensavers. 
Animation Camera with target: 

An Animation Camera with Target always looks at a dummy object. Both the camera and the 
helper object can be animated. 




Object Inspection Camera: 

Quest3D is the perfect product visualisation package. An Object Inspection Camera can orbit 

around a 3D model, enabling the user to view the object from all angles. Such 'products' 

might be as small as technological gadgets, or as large as entire cityblocks. 

1st Person Walkthrough Camera: 

It is often desirable, and rewarding, to experience an architectural visualisation as if you 

were there. A 1st Person Walkthrough Camera presents a scene at regular eye level, and allows 

the user to walk around freely. Since the view is close to real life, this type of camera 

provides the highest level of immersion in a scene. Many computer games make use of the 

1st person perspective. 

3rd Person Walkthrough Camera: 

In contrast to the 1st person point of view, a 3rd person perspective places the camera just 

outside of a virtual character. The 3rd Person Walkthrough Camera is attached to this 

character, or avatar, and follows it wherever it goes. 

Every kind of camera is defined by particular characteristics: 

zoom factor: 

The zoom factor is a quick way to zoom a camera in or out. 

The default value of '1.0' results in a normal view. A higher Zoom Factor brings the scene 

closer, a lower value sets it further away. 



Modeling Virtual Reality Web Application 



255 



clipping planes: 

The clipping planes of a camera are virtual boundaries between which a scene is rendered. 

Polygon surfaces outside of these boundaries are not calculated. 

f°2L 

Fog can be used to blend various elements in a virtual environment together. It can add to 

the mood of a scene and suggest depth. 

Fog can be used to blend various elements in a virtual environment together. It can add to 

the mood of a scene and suggest depth. A bonus of using fog is that the far clipping plane of 

the camera may be decreased to just behind the Fog End value. If used right, this will result 

in a smooth transition between the scene and the "nothing" beyond the far clipping plane. 



Camera 



Near 


Far 


clipping 


dipping 


plane 


plane 










Mffc 


1 
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11. Sounds 

From simple menu click sounds to full ambient music, audio can really add to a scene. It 

helps to set the mood, just as sharp graphics and lively animation can. The use of spoken 

voice samples can help users to better understand an application. Turn off the sound while 

watching television or playing a computer game, and youTl instantly miss the richness of 

audio. 

Sound can be stored digitally on the computer, in many different formats. Quest3D supports 

both '.wav' and '.mp3' files. 

The Sound File channel is used to store .wav samples within Quest3D. Its properties window 

contains a number of options including icons to load, play, stop and pause a sample. As the 

name suggests, the Repeat option allows for continuous playback. 



Sound File Properties 



a; | ► | ■ ii 

-General Properties 
Speed [%} 

Volume [%} 




Panning \Z] 
Repeat 



1 50. 

r 



3D Positioning 


| None 


d 




of full volume. 


Silent at 1/| 100. 
Halve volume every 


| 1- units. 






Ok 



Speed ranges from 0% to 200%, and volume ranges from 0% to 100%. Both can be 

dynamically changed through their respective child links. 

Panning controls how the sound is divided between the left and right speaker: 0% means 

only the left one is used, 100% means the right one. 50% (default value) plays the sound at 

equal levels through both speakers. 

The Sound Command channel allows you to control all aspects of a sample, such as playback 

and volume. It also allows for loading '.wav' files from harddisk. The drop-down list 

presents all the various options. A Sound Command must be called once to be executed, and 

affects all its children of the Sound File channel type. 

Besides simply playing a sound, it is also possible to place it in 3D space. 3D Positioned 

sound requires a Listener channel. It is usually connected to the location of the virtual 

camera, and automatically adjusts volume, panning and frequency by calculating a sound's 

relative position and distance. Only Mono sound samples can be positioned in 3D space. 
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The 3D Positioning options of the Sound File channel define the quality of the simulation. 

Higher quality requires more cpu power. 

The '.mp3' format is also supported in Quest3D. The MP3 File channel is used to store sound 

samples of this type. 

Through its properties window it is possible to load a file from disk. The Save File in 

Channelgroup option allows you to store the sample data inside of the channel itself. This 

will of course increase the file size of the channelgroup. The MP3File's only child may also 

contain a file name in the form of a Text channel. 

The MP3 Control channel can be used to control playback and volume. It can also be used to 

retrieve information on total length and current playing position, and to skip to a specific 

position in the sample. 




12. Particle sistems 

Many special effects in movies and games, such as fire, smoke, explosions and all kinds of 
magical effects, have one thing in common: they are all created using so called particle 
systems. 
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A particle system is a collection of objects released from an emitter and usually moved and 

changed over time. Particles are square objects facing the camera. 

For every particle in the system, the position, rotation and size is stored. In addition, other 

attributes such as speed, color and transparency may be used. Forces such as gravity may be 

applied to all particles in the system. 

Particles are released from an emitter. In Quest3D, this is in the form of an 3D Object Data 

channel. 

The particle system cycles through all of the vertices in the object and constantly emits a 

particle from one of these positions. 

By randomly placing a number of vertices in 3D space, it appears as if the particles are 

released randomly as well. This produces convincing results for special effects such as fire 

and smoke. 




A Particle Flow Grid channel can be used to control the flow of the system. It divides a 
collision object into several parts and calculates forces inside of each of these. Forces can be 
created by the collision object, external objects and the particles themselves. 
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13. Mathematical operators 

Values are at the heart of any Quest3D project. They are used to define an object's color, position 
and size for example. Values can also describe scores, program settings, menu option etc. 
A simple instruction which describes mathematical expression is Set Value, which realizes a 
simple value association: 




Old Value 




In most practical cases Set Value channels need to be carried out only once. Therefore, they 
are often called by Trigger channels. 

Mathematical expressions can be described in Quest3D using the Expression Value channel. It 
serves as an advanced calculator. Its properties window contains a box for the actual 
formula, and help text describing all the usuable variables and operators. 



IMl-UMA'milJ 



! 



Current Value- 



Expression status- 
Invalid Formula 



Expression- 



Formula Variables — 

OLD Current value 
A : No Channel Linked 
B : No Channel Linked 
C : No Channel Linked 
D : No Channel Linked 
E : No Channel Linked 
etc . . 

PI: 3.141592 653 589793 2 3 84 62 643 3 83 2 795 
RAND: Random Value between 0.0 and 1.0 
TC : Tickcount of current frame never bigger then 2 
Supported Functions — 



AES() ACOS{) ADD() ASIN() ATAN ( ) AVG { ) CQS{] COSH() EXP { ) FLOOR {) FRAC() 
LOG2() LOG10() MAX{) HAXDIF{) HIN ( ) ROUND ( ) SIN() SINH{) SQRT() SUM ( ) 
TAN ( ) TANH { ) 
+ - / | £ ~ ** % «»<=>=<> == != ! ? A 



.annel to formula 



OK 



Cancel 
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Values connected to the child links of the Expression Value can be used as part of the 
expression within the channel as well. The first child is presented by the letter 'A', the 
second by the letter 'B', and so forth. 




■-Expression— 



Value 



Multiple expressions can be linked to one another within the same channel using the logical 

operators AND ( && ) and OR ( | | ). 

In this example the condition is met only if A==l && B==2. 




In this other case, the expression A==B ? CD is based on identity between A and B. 




262 



Advances in Robotics, Automation and Control 



Mathematical functions are realized throught the Envelopes, main instruments for managing 

animations. 

For advanced simulations and models, actual formula's may be used by inserting a number of 

coordinate pairs into the graph. Quest3D will handle the interpolation between these points. 



mn 



•'^Create ^5edect -fTMove ^Delete Uievusel !e lilall keys. 
Kw I 

|DelKQvl|HCWK^| 



I-/"- 



lAl±\ 



HaizviMr] |-3 



|TCB 


d 


|tJ|c|b)bH 




JDrfait 


zl 



| | Ajj^l E Mma <^| [x"[y1 |zoam » j|xl7] 




Ok Carol | 



This Envelope graph show the formula y = x 2 



14. Lua Scripting 

Quest3D channels are building blocks containing precompiled C++ and DirectX code. The 

entire set of channels provides functionality for many aspects of real-time 3D development. 

Despite the open and logical structure of the channels system, certain functionality can be 

handled more efficiently or more easily in script language. 

Quest3D supports Lua, a free third party scripting environment. 

Lua in Quest3D is especially applicable for cases such as loading and unloading channel 

groups, complex calculations and iterative structures (for loops). 



i Bmaaaa 



^jnjx] 



001 
002 
003 
004 
005 
006 
007 



— CallChannel function is called when channel is calif 
function CallChannel ( ) 

end 

— Get Value function is called when this channels is ui 
function Get Value ( ) 

return -1 
end 



A 



Last error message- 
No error to report 



Lua channels can have one of two functions, or both: 
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• The first is CallChannel, and is executed when the channel is simply called; 

• The second is GetValue, and is executed when the Lua channel is used as a value. The 
actual value of the Lua channel itself is equal to the value the script 'returns' at the end 
of the function. 

Just like in other programming and scripting languages, variables used in a Lua script must 
be declared. This statement assigns memory to the variable. The word local means that the 
variable is only used in one particular place, in this case the Lua channel. As soon as the 
script is done, the memory is cleared again. 

local variable 

local variable = 1 

variable = variable + 1 

Quest3D supports the following standard Lua group functions: base, string and math. 

local variable = math.cos(value) 

The first example calculates the cosine of the value between the brackets. 




Any number of children of any type may be connected to a Lua channel. 

Children may be accessed in a Lua script using the appropriate function. 

As usual, they have to be declared first. The statement below accesses the first child 

connected to the Lua channel (at position '()'). 

local variable = channel. GetChild(O) 

The actual value of a Value channel connected to a Lua channel may be accessed using the 
following commands. 

local variable = channel. GetChild(O) 

local value = variable:GetValue() 

Text can also be retrieved, using the following statements: 

local variable = channel. GetChild(O) 

local text = variable:GetText() 

The value or text of a child may be set using the following commands: 



264 Advances in Robotics, Automation and Control 

variable:SetValue(value) 
variable: SetText(text) 

Note the difference between using functions of predefined and local structures. For 
predefined structures such as 'charmer, a function is preceded by a '/ (dot). For local 
structures such as variables, a function is preceded by a ':' (double colon punctuation mark). 
Functions to load or unload channel group are very usefull. 
For the LUA group q, the following instructions realize these statments: 

q.LoadChannelGroupC'group.cgr", "PoolName", instance) 

q.RemoveChannelGroupfPoolName", instance) 

15. Product publications 

At a certain point, a Quest3D project may be finished and ready for distribution. Whether it 

is a modest free screensaver or an extensive commercial training application, the Quest3D 

program must be exported out of the editor and saved into a format the target audience can 

run. 

Different options are available to publish a Quest3D project: 



Can publish to type 

.q3d player file 

Web application 

Screensaver 

WinAmp plug-in 



Stand-alone self -running .exe 
Installer file 



Internet ActiveX: .q3d and .htm: 

files of type .q3d can also be viewed in an internet browser that supports ActiveX. The .q3d 

file must be linked from a regular webpage to show. Web pages containing Quest3D media 

elements will automatically try to download and install the Quest3D Viewer Package from 

the Quest3D.com website (www.quest3d.com). 

Stand-alone programs: .exe: 

a Quest3D project can also be published to a stand-alone self-executing .exe file. Besides 

DirectX, this type of publication requires no additional components. 

Installation files: .exe: 

installers are .exe archive files that copy and expand a program to a specified directory on a 

harddisk. 

16. References 

www.quest3d.com 
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1. Introduction 

An outlier is an observation (or measurement) that is different with respect to the other 
values contained in a given dataset. Outliers can be due to several causes. The measurement 
can be incorrectly observed, recorded or entered into the process computer, the observed 
datum can come from a different population with respect to the normal situation and thus is 
correctly measured but represents a rare event. In literature different definitions of outlier 
exist: the most commonly referred are reported in the following: 

"An outlier is an observation that deviates so much from other observations as to 

arouse suspicions that is was generated by a different mechanism " (Hawkins, 1980). 

"An outlier is an observation (or subset of observations) which appear to be 

inconsistent with the remainder of the dataset" (Barnet & Lewis, 1994). 

"An outlier is an observation that lies outside the overall pattern of a distribution" 

(Moore and McCabe, 1999). 

"Outliers are those data records that do not follow any pattern in an application" (Chen 

and al, 2002). 

"An outlier in a set of data is an observation or a point that is considerably dissimilar or 

inconsistent with the remainder of the data" (Ramasmawy at al., 2000). 
Many data mining algorithms try to minimize the influence of outliers for instance on a final 
model to develop, or to eliminate them in the data pre-processing phase. However, a data 
miner should be careful when automatically detecting and eliminating outliers because, if 
the data are correct, their elimination can cause the loss of important hidden information 
(Kantardzic, 2003). Some data mining applications are focused on outlier detection and they 
are the essential result of a data-analysis (Sane & Ghatol, 2006). 

The outlier detection techniques find applications in credit card fraud, network robustness 
analysis, network intrusion detection, financial applications and marketing (Han & Kamber, 
2001). A more exhaustive list of applications that exploit outlier detection is provided below 
(Hodge, 2004): 

Fraud detection: fraudulent applications for credit cards, state benefits or fraudulent 

usage of credit cards or mobile phones. 

Loan application processing: fraudulent applications or potentially problematical 

customers. 

Intrusion detection, such as unauthorized access in computer networks. 
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Activity monitoring: for instance the detection of mobile phone fraud by monitoring 

phone activity or suspicious trades in the equity markets. 

Network performance: monitoring of the performance of computer networks, for 

example to detect network bottlenecks. 

Fault diagnosis: processes monitoring to detect faults for instance in motors, generators, 

pipelines. 

Structural defect detection, such as monitoring of manufacturing lines to detect faulty 

production runs. 

Satellite image analysis: identification of novel features or misclassified features. 

Detecting novelties in images (for robot neotaxis or surveillance systems). 

Motion segmentation: such as detection of the features of moving images 

independently on the background. 

Time-series monitoring: monitoring of safety critical applications such as drilling or 

high-speed milling. 

Medical condition monitoring (such as heart rate monitors). 

Pharmaceutical research (identifying novel molecular structures). 

Detecting novelty in text. To detect the onset of news stories, for topic detection and 

tracking or for traders to pinpoint equity, commodities. 

Detecting unexpected entries in databases (in data mining application, to the aim of 

detecting errors, frauds or valid but unexpected entries). 

Detecting mislabeled data in a training data set. 
How the outlier detection system deals with the outlier depends on the application area. 
A system should use a classification algorithm that is robust to outliers to model data with 
naturally occurring outlier points. In any case the system must detect outlier in real time and 
alert the system administrator. Once the situation has been handled, the anomalous reading 
may be separately stored for comparison with any new case but would probably not be 
stored with the main system data as these techniques tend to model normality and use 
outliers to detect anomalies (Hodge, 2004). 

2. Traditional approaches 

The salient traditional approaches to outlier detection can be classified as either distribution- 
based, depth-based, clustering, distance-based or density-based. 

2.1 Distribution-based method 

These methods are typically found in statistics textbooks. They deploy some standard 
distribution model (Normal, Poisson, etc.) and flag as outliers those data which deviate from 
the model. However, most distribution models typically apply directly to the future space 
and are univariate i.e. have very few degrees of freedom. Thus, they are unsuitable even for 
moderately high-dimensional data sets. Furthermore, for arbitrary data sets without any 
prior knowledge of the distribution of points, expensive tests are required to determine 
which model best fits the data, if any! (Papadimitriou et al., 2002) Fitting the data with 
standard distributions is costly and may not produce satisfactory results. The most popular 
distribution is the Gaussian function. (Kim & Cho, 2006). 

A method was proposed by Grubbs which calculates a Z value as the difference between the 
mean value for the attribute and the query value divided by the standard deviation for the 
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attribute, where the mean and the standard deviation are calculated from all attribute values 
including the query value. The Z value for the query is compared with a 1% or 5% 
significance level. The technique requires no pre-defined parameters as all parameters are 
directly derived from the data. However, the success of this approach heavily depends on 
the number of exemplars in the data set. The higher the number of records, the more 
statistically representative the sample is likely to be (Grubbs, 1969). 

A Gaussian mixture model (GMM) and computed outlierness was proposed based on how 
much a data point deviates from the model (Roberts & Tarassenko, 1995). 
The GMM is represented by equation (1): 

M 

P(t|x)=Xaj«Jj(t|x) (1) 

j=l 

where M is the number of kernels (^), aj(x) the mixing coefficients, x the input vector and t 

the target vector. 

A Gaussian probability density function is defined by equation (2): 
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where d is the dimensionality of input space, a is the smoothing parameter, //.(*) 
represents the centre of ;th kernel and <j 2 .{x) is the variance. 

Another model was proposed by Laurikkala et al.(Laurikkala et al., 2000) in probably one of 
the simplest statistical outlier detection techniques. This method uses informal box plots to 
pinpoint outliers in both univariate and multivariate datasets, produces a graphical 
representation and allows a human auditor to visually pinpoint the outlying points. This 
approach can handle real-valued, ordinal and categorical attributes. 

Distribution based methods have some advantages such as mathematical justification and 
fast evaluation once they are built. However, they also have important drawbacks, such as 
the need for assuming a distribution and a considerable complexity for high dimensional 
problems. 

Indeed statistical models are generally suited to quantitative real-valued data sets at the 
very least quantitative ordinal data distributions, where the ordinal data can be transformed 
into suitable numerical values through statistical processing. This fact limits their 
applicability and increases the processing time if complex data transformations are 
necessary before processing (Hodge, 2004). 

2.2 Depth-based method 

This approach is based on computational geometry and computes different layers of k-d 
convex hulls (Johnson et al., 1998). Based on some definition of depth, data objects are 
organized in convex hull layers in data space according to peeling depth and outliers are 
expected to be found from data objects with shallow depth values. In theory, depth-based 
methods could work in high dimensional data space. However, due to relying on the 
computational of k-d convex hulls, these techniques have a lower bound complexity of 
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Q(N k / 2 ), where N is number of data objects and k is the dimensionality of the dataset. This 
makes these techniques infeasible for large dataset with high dimensionality (He et aL, 
2002). 

2.3 Clustering 

Clustering is a basic method to detect potential outliers. From the viewpoint of a clustering 
algorithm, potential outliers are data which are not located in any cluster. Furthermore, if a 
cluster significantly differs from other clusters, the objects in this cluster might be outliers. A 
clustering algorithm should satisfy three important requirements (Birant & Kut, 2006): 

Discovery of clusters with arbitrary shape; 

Good efficiency on large databases 

Some heuristics to determine the input parameters. 
Fuzzy c-means (FCM) is a method of clustering which allows one piece of data to belong to 
two or more clusters. This method (developed by Dunn in 1973 and improved by Bezdek in 
1981) is based on minimization of the following objective function: 

where C is the total number of clusters, N is the total number of data, m is any real number 
greater than 1, Uij is the degree of membership of x\ to the ;-th cluster, x\ is the z'-th of the d- 
dimensional measured data, q is the ^-dimensional center of the cluster, and ||*|| is any 

norm expressing the similarity between any measured data and the center. 
Fuzzy partitioning is carried out through an iterative optimization of the above cited 
objective function, with the update of membership degree Uij and the cluster centers q 
respectively given by: 
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The iteration stops when the following condition holds: 

max..{|«<* +1 > -««|}< £ (6) 

where 8 is a termination criterion lying in the range [0, 1] whereas k is the iteration step. 
This procedure converges to a local minimum or a saddle point of J m . 

A quite popular technique used for outlier detection is the introduction of a new variable 
called credibility of a vector. This method is proposed by Chintalapudi and Kam 
(Chintalapudi & Kam, 1998) with regard to the above described fuzzy clustering algorithms. 
Credibility measures the "how typical" the vector is with respect to the entire data set. An 



Outlier Detection Methods for Industrial Applications 269 

outlier is expected to have a low value of credibility compared to a non-outlier. The use of 
the new variable leads to the Credibilistic Fuzzy C Means algorithm. Two formulations are 
made for the credibility variable, based on the mean nearest statistical distance. Simulations 
demonstrate that the proposed schemes are robust with respect to the presence of outliers 
and with respect to the parameter values needed to implement the algorithm. 
Most clustering algorithms, especially those developed in the context of Knowledge 
Discovery in Databases (KDD) (e.g. CLARANS (Ng&Han, 1994), DBSCAN (Ester et al, 
1998), BIRCH (Zhang et al, 1996), STING (Wang et al, 1997), Wave Cluster (Sheikholeslami 
et al, 1998), DenClue (Hinneburg&Keim, 1998), CLIQUE (Aggarwal et al, 1998) , OPTICS 
(Ankerst et al.,1999), PROCLUS (Aggarwal et al, 1999)) are to some extent capable of 
handling exceptions. 

However, since the main objective of a clustering algorithm is to find clusters, they are 
developed to optimize the outlier detection. The exceptions (called "noise" in the context of 
clustering) are typically just tolerated or ignored when producing the clustering result. Even 
if the outliers are not ignored, the notions of outliers are essentially binary and there are no 
qualification as to how outlying an object is (Breunig et al., 2000). 

2.4 Distance-based method 

The notion of distance-based (DB) outlier is been defined by Knorr and Ng (1988): 

An object O in a dataset T is a DB(p,D)-outlier if at least fraction p of the objects in T lie greater than 

distance D from O. 

The concept of DB-outlier is well defined for any dimensional dataset. The parameter p is 

the minimum fraction of objects in a data space that must be outside an outlier D- 

neighborhood (Li & Hiroyuki, 2007). 

This notion generalizes many concepts from distribution-based approach and better faces 

computational complexity. It is further extended based on the distance of a point from its k- 

th nearest neighbor (Ramasmamy et al., 2000). 

After ranking points by the distance to its k-th nearest neighbor, the top k points are 

identified as outliers. Alternatively, in the algorithm proposed by Angiulli and Pizzuti 

(Angiulli & Pizzuti,2000), the outlier factor of each data point is computed as the sum of 

distances from its k nearest neighbors. 

Both the method proposed in (Matsumoto et al., 2007) and the Mahalanobis outlier analysis 

(MOA) (Marquez et al., 2002) are distance-based approaches which exploit Mahalanobis 

distance as outlying degree of each data point. 

In 1936 P.C. Mahalanobis introduced a distance measure (Mahalanobis, 1936) which is based 

on correlations between variables by which different patterns can be identified and analyzed 

and provides a useful way of determining similarity of an unknown sample set to a known 

one. It differs from Euclidean distance in that it takes into account the correlations of the 

data set and is scale-invariant, i.e. not dependent on the scale of measurements. 

Formally, the Mahalanobis distance from a group of values with mean u=( ui, U2, U3, ..., u p ) T 

and covariance matrix Z for a multivariate vector x=( xi, X2, X3, . . ., x p ) T is defined as: 

D M =J(x-ju) T T- l (x- M ) (7) 

Mahalanobis distance can also be defined as dissimilarity measure between two random 
vectors x and y of the same distribution with the covariance matrix P: 
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d(x,y) = ^x-y) T p-\x-y) (8) 

If the covariance matrix is the identity matrix, the Mahalanobis distance reduces to the 
Euclidean distance. Finally if the covariance matrix is diagonal, then the resulting distance 
measure is called the normalized Euclidean distance: 



d(x,y)=\Yp^- (9) 



where a i is the standard deviation of the Xi over the sample set. 

Mahalanobis distance is computed on the basis of the variance of data points. It describes 
the distance between each data point and the center of mass. 

When one data point is on the center of mass, its Mahalanobis distance is zero, and when 
one data point is distant from the center of mass, its Mahalanobis distance is more than zero. 
Therefore, datapoints that are located far away from the center of mass are considered 
outliers (Matsumoto et al, 2007). 

2.5 Density-based method 

This method assigns to each object a degree to be an outlier. This degree is called the local 

outlier factor (LOF) of an object. It is "local" in the sense that the degree depends on how 

isolated the object is with respect to the surrounding neighborhood. 

In LOF algorithm, outliers are data objects with high LOF values whereas data objects with 

low LOF values are likely to be normal with respect to their neighborhood. High LOF is an 

indication of low-density neighborhood and hence high potential of being outlier (Mansur & 

Noor, 2005). In order to understand LOF method, it is necessary to define some auxiliary 

notions. (Breunig et al., 2000). 

Definition 1: (k-distance of an object p). 

For any positive integer k, the k-distance of object p, denoted as k-distance (p), is defined as 

the distance d(p,o) between p and an object oGD such that: 

i. For at least k objects o'G D\ {p} it holds that d(p,o')<d(p,o) and 

ii. For at most k-1 objects o'G D\ {p} it holds that d(p,o')<d(p,o). 

Definition 2: (k-distance neighborhood of an object p). 

Given the k-distance of p, the k-distance neighborhood of p contains every object whose 

distance from p is not greater than the k-distance, i.e. 

N k . dlstance(p) (p) = N k (p) = {qeD\ {p} | d(p,q) <k- distmce(p)} (10) 

These objects q are called the k-nearest neighbors of p. 

Definition 3: (reachability distance of an object p w.r.t. object o). 

Let k be a natural number. The reachability distance of object p with respect to object o is 

defined as 

reach - dist k = max [k - dis tan ceip), d(p, o)} (11) 

The higher the value of k, the more similar the reachability distances for objects within the 
same neighborhood. 
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In a typical density-based clustering algorithm, there are two parameters that define the 
notion of density: a parameter MinPts specifying a minimum number of objects and a 
parameter specifying a volume. These two parameters determine a density threshold for the 
clustering algorithms to operate. That is, objects or regions are connected if their 
neighborhood densities exceed the given density threshold. To detect density-based outliers, 
however, it is necessary to compare the densities of different sets of objects, which means 
that the density of sets of objects must be dynamically determined. An idea is to keep 
MinPts as the only parameter and use the values reach-distMinPts(p,o), for o GNMinPts(p). 
Definition 4: (local reachability density of an object p). 
The local reachability density of p is defined as: 

lrd MinPts (P) = V T^F~, ~( 7 ( 12 ) 

Z, reach - dlSt MinPts(P>°) 

O^ ' MinPts (P) 

\^ MinPts \P) | 

Intuitively, the local reachability density of an object /? is the inverse of the average 
reachability distance based on the MinPts-nearest neighbors of p. 
Definition 5: (local outlier factor of an object p) 
The local outlier factor of p is defined as: 

y lrd MinPt5 (o) 

T r>T7 ( n\ oeN MinPt s (P) I™ MinPts \P) /io\ 

LUt MinPts \P) = — —, (- 1 6 ) 

\N MinPts (P)\ 

The outlier factor of object p captures the degree to which we call p an outlier. It is the 

average of the ratio of the local reachability density of p and those of p's MinPts-nearest 

neighbors. 

It is possible to know the range where LOF lies. By defining £ as 

max [reach - dist(p, q)) 
min [reach - dist(p, q)} - 1 

it can be demonstrated that (Breunig et al., 2000): 

— !— < LOF(p) <\ + s (15) 

1 + s 

3. Wavelets-based outlier detection approach 

The wavelet transform or wavelet analysis is probably one of the most recent solutions to 
overcome the shortcomings of the Fourier transform. In wavelet analysis the use of a fully 
scalable modulated window solves the signal-cutting problem. The window is shifted along 
the signal and for every position the spectrum is calculated. This process is repeated many 
times with a slightly shorter (or longer) window for every new cycle. The processing result 
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is a collection of time-frequency representations of the signal corresponding to different 

resolutions. 

The wavelet transform is an operation that transforms a function by integrating it with 

modified versions of some kernel function. (Combes et al., 1989). The kernel function is 

called the mother wavelet, and the modifications are translations and compressions of the 

mother wavelet. A function g(t) can be a mother wavelet if it is admissible, i.e. if the following 

condition holds (Grossmannn & Morlet, 1984): 

t»\GW^ a<<a ( 1 6) 



where G(co) is the Fourier transform of g(t). The constant ^represent the admissibility of the 
function that must be finite for inversion of the wavelet transform. Any admissible function 
can be a mother wavelet (Weiss, 1994). 

Wavelet transforms are classified into discrete wavelet transforms and continuous wavelet 
transforms on the basis of the applications and data. 

Outlier detection by means of the wavelet transform is a recent study area. (Bruce et al., 
1994) (Wang, 1995) (Dorst, 1999) and can be an alternative to the previously discussed 
methods because the data do not need to belong to a known distribution and the 
discontinuities in the processed data, that can often correspond to outliers, are easily 
detected. (Kern et al, 2005). 

For instance Yu et al. (Yu et al., 2002) introduced a new outlier detection approach, called 
FindOut (which stands for Find Outliers), based on wavelet transform. FindOut identifies 
outliers by removing clusters from the original data. The main idea is to apply signal 
processing techniques to transform the space and find the dense regions in the transformed 
space. The remaining objects in the non-dense regions are labeled as outliers. The primary 
motivation for applying signal-processing techniques to spatial datasets comes from the 
observation that the multidimensional data points can be represented as a d-dimensional 
signal. Wavelet transform is a signal processing technique that decomposes a signal into 
different frequency sub-bands; this method use two filters: a high-pass filter and a low-pass 
filter. One important step of wavelet transform is to convolve its input with a low-pass filter 
that has the main property of removing noise (outlier). The low-pass filter removes the 
outliers and smoothes the input data. However in FindOut the objective is to find outliers 
and not remove them so the idea is to remove the clusters from the original data and thus 
identify the outliers. 

4. Outlier detection using artificial intelligence techniques 

When dealing with industrial automation, where data coming from the production field are 
collected with different and heterogeneous means, the occurrence of outliers is more the rule 
than the exception. Standard outlier detection methods fail to detect outliers in industrial 
data because of the high dimensionality of the data. In these cases, the use of artificial 
intelligence techniques has received increasing attention in the scientific and industrial 
community, as the application of these techniques shows the advantage of requiring poor or 
no a priori theoretical assumption on the considered data. Moreover their implementation is 
relatively simple and with no apparent limitation on the dimensionality of the data. 
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4.1 Neural networks 

In 1943, McCulloch and Pitts introduced the idea of an artificial neuron to process data. In 
the 50ies this work was advanced by arranging neurons in layers. Although learning rules to 
cope with multiple layers of perceptrons were not developed until later, this work formed 
the basis of the Multi-Layer Perceptron (MLP) that is used today. 

Another kind of neural network that is frequently used is the Radial Basis Function (RBF), 
which exploits gaussian activation functions in the first (or sometimes called hidden) layer. 
Once the inputs and the outputs have been defined, it is useful to see if the data set contains 
any points that violate this limits. If there are many similar examples for a given input 
pattern, an outlier can be classified as the one which is furthest from the median value. 
Other methods that can be applied to detect outliers are the Principle Component Analysis 
(PC A) and Partial Least Squares (PLS). Outliers can be found by investigating points at the 
edges of the previously created clusters. (Tenner et al., 1999). 

Liu and Gader indicated that including outlier samples in training data and using more 
hidden nodes than required for classification for MLP and BRF networks and proceeding an 
RBF with principal Component decomposition can achieve outlier rejection. The further 
addition of a regularization term to the PCA-RBF can achieve an outlier rejection 
performance equivalent or better than that of other networks without training on outliers 
(Liu & Gader, 2000). 

Williams et al. propose to apply the so-called Replicator Neural Network (RNN) for outlier 
detection. RNN are multi-layer perceptron neural networks with three hidden layers and 
the same number of output neurons and input neurons to model the data. The input 
variables are also the output variables so that the RNN forms compressed model of data 
during training. A measure of outlyingness of individuals is developed as the reconstruction 
error of individual data points. (Hawkins et al., 2002). This method is often compared with 
other methods, in particle with Hadi94 (Hadi, 1994) and Donoho-Stahel (Knorr et al., 2001). 
The Hadi94 is a parametric bulk outlier detection method for multivariate data and Donoho- 
Stahel uses the outlyingness measure compute by the Donoho-Stahel (Rousseeuw & Leroy, 
1997) estimator of location and scatter. These two methods perform well on large and 
complex data sets. However these are parametric methods and lack of the flexibility of non- 
parametric methods, such as Minimum Message Length (MLL) (Oliver et al., 1996) and 
RNN. MLL clustering works well for scattered outlier while RNN degrades with datasets 
containing radial outliers. However RNN performs satisfactory for small and large datasets 
(Neural Network methods often have difficulty with such smaller datasets) (Williams et al., 
2002). 

The self-organizing map (SOM) is an artificial neural networks, which is trained by using 
unsupervised learning in order to produce a low dimensional representation of the training 
samples while preserving the topological properties of the input space. (Munoz & 
Muruzabal, 1997). 

Nag et al. (Nag et al., 2005) proposed a SOM based method for outlier detection, which 
identifies the multidimensional outliers and provides information about the entire outlier 
neighborhood. The SOM based outlier detection method is non-parametric and can be used 
to detect outliers from large multidimensional datasets. This method has the advantages 
that does not require any a priori assumption on the variable, is easy to implement and does 
not have problems with dimensionality of data. 
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4.2 Support Vector Machine (SVM) 

Support Vector Machine, introduced by V. Vapnik (Vapnik, 1982), is a method for creating 
functions from a set of labelled training data. The function can be a classification function or 
a general regression function. In classification tasks, SVMs operate by finding a 
hypersurface in the space of the possible inputs, which separates the samples belonging to 
different classes. Such division is chosen so as to have the largest distance from the 
hypersurface to the nearest of the positive and negative examples (Vapnik, 1998). 
Jordaan and Smits (Jordaan & Smits, 2004 ) propose a robust model-based outlier detection 
approach that exploits the characteristics of the support vectors extracted by the SVM 
method (Cortes & Vapnik, 1995). This method makes use of several models of varying 
complexity to detect outliers based on the characteristics of the support vectors obtained 
from SVM-models. This approach has the advantage that the decision does not depend on 
the quality of a single model, which adds to the robustness of the approach. Furthermore, 
since it is an iterative approach, the most severe outliers are firstly removed. This allows the 
models in the next iteration to learn from "cleaner" data and thus reveal outliers that were 
masked in the initial model. The need for several iterations as well as the use of models, 
however, makes the on-line application of this method difficult for the not negligible 
computational burden. Moreover, if the data to be on-line processed come from dynamic 
systems, which tend to change (more or less rapidly) their conditions through time, the 
models update is required and this can furtherly slow the outlier detection. 

4.3 Fuzzy logic 

Fuzzy Logic (FL) is linked with the theory of fuzzy sets, a theory which relates to classes of 
objects with un-sharp boundaries in which membership is a matter of degree. 
Fuzzy theory is essential and is applicable to many systems — from consumer products like 
washing machines or refrigerators to big systems like trains or subways. Recently, fuzzy 
theory has been a strong tool for combining new theories (called soft computing) such as 
genetic algorithms or neural networks to get knowledge from real data (Melin & Castillo, 
2008). 

Fuzzy logic is conceptually easy to understand, tolerant of imprecise data and flexible. 
Moreover this method can model non-linear functions of arbitrary complexity and it is based 
on natural language. Natural language has been shaped by thousands of years of human 
history to be convenient and efficient. Since fuzzy logic is built atop the structures of 
qualitative description used in everyday language, fuzzy logic is easy to use (Baldwin, 1978). 
Fuzzy inference system (FIS) (Ross, 2004) is the process of formulating the mapping from a 
given input to an output using fuzzy logic. The mapping then provides a basis from which 
decisions can be made, or pattern discerned. The process of fuzzy inference involves: 
membership functions (MF), a curve that defines how each point in the input space is 
mapped to a membership value or degree of membership between and 1; fuzzy logic 
operators (and, or, not); if-then rules. Since decisions are based on the testing of all of the 
rules in an FIS, the rules must be combined in some manner in order to make decision. 
Aggregation is the process by which the fuzzy sets that represents the outputs of each rule 
are combined into a single fuzzy set. Aggregation only occurs once for each output variable, 
just prior to the final step, defuzzification (Patyra & Mlynek, 1996). 

Due to the linguistic formulation of its rule basis, the FIS provides an optimal tool to 
combine more criteria among those that were above illustrated according to a reasoning that 
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is very similar to the human one. So doing, in practical application, the knowledge of the 
technical expert personnel can easily be exploited by the system designer. 
In the following, an exemplar method will be proposed which exploits a fuzzy inference 
system (FIS) in order to combine several outliers detection strategies by jointly evaluating 
four features of a particular datum within a series of measurements (Cateni et al., 2007). 
For each pattern the following four features are extracted and fed as input of the FIS: 

Distance between each element and the centroid of the overall distribution normalized 

with respect to the average value, (dist) 

Fraction of the total number of elements that are near to the pattern itself, (n-points) 

Mean distance between the considered pattern and the remaining patterns normalized 

with respect to the maximum value, (memb-deg) 

Degree of membership of the patterns to the cluster to which it has been assigned by the 

preliminary fuzzy c-means clustering stage, (mean-dist) 
In geometry the centroid of an object X in n-dimensional space is the intersection of all 
hyperplanes that divide X into two parts of equal moment about the hyperplane. Roughly 
speaking, the centroid is a sort of "average" of all points of X. 

The FIS is of Mandami type (Mandami & Assilian, 1975) and the FIS output variable, named 
outlier-index (outindx), is defined in the range [0;1]. The output function provides an 
indication on the risk that the considered pattern is an outlier. 

The inference rules relating the output variable to the four inputs is formulated through a 
set of 6 fuzzy rules, that are listed below: 

1. IF (dist is very high) AND (n-points is very small) AND (memb-deg is low) AND 
(mean-dist is big) THEN (outindx is very high). 

2. IF (dist is medium) AND (n-points is small) AND (memb-deg is quite low) AND 
(mean-dist is small) THEN (outindx is quite high). 

3. IF (dist is low) AND (n-points is medium) AND (memb-deg is quite low) AND (mean- 
dist is very small) THEN (outindx is low). 

IF (dist is medium) AND (n-points is very small) AND (memb-deg is quite low) AND 
(mean-dist is small) THEN (outindx is quite high). 

5. IF (dist is low) AND (n-points is small) AND (memb-deg is high) AND (mean-dist is 
quite big) THEN (outindx is low). 

6. IF (dist is low) AND (n-points is medium) AND (memb-deg is high) AND (mean-dist is 
small) THEN (outindx is low). 

Figure 1 depicts a scheme of the proposed method. An outlier is detected when its outlier 
index overcome a prefixed threshold. 
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Fig. 1. Block diagram depicting the overall system for outliers detection. 
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5. Results 

The proposed outlier detection method, which does not require a priori assumption on the 
data, has been tested in the preprocessing of data provided by a steelmaking industry, 
where outliers can provide indications on malfunctionings or anomalous process conditions. 
The method has been tested by considering that the technical personnel provided 
indications of those values that should be considered as outliers. 

In this work two applications are proposed that use two different variables that are 
important to determine the quality of steel and the final destination. The variables are 
composed by 100 samples normalized respect their mean value. 

The performance of the fuzzy logic-based method has been compared with Grubbs test and 
Local Outlier Factor (LOF) techniques, that are considered among the most important and 
widely adopted traditional outlier detection methods. The results show that the fuzzy logic- 
based method outperforms the other approaches, but, on the other hand, the required 
computational time is approximately ten times greater than the time required by traditional 
methods, due to the increased complexity of the FIS-based evaluation. 

In particular, in the first exemplar application, the considered variable represents the 
concentration of a chemical element extracted from the analysis made on molten steel. In 
this piece of database there are five outliers. The samples that are considered outliers are 3, 
8, 16, 35 and 92 referring respectively to following values: 

0.9349 0.9385 1.0455 0.9626 0.9541. 

Figure 2 shows the result of Grubbs test. It clearly appears that only two anomalous 

samples have been recognized as outliers (the 3 rd and 8 th samples). 

Figure 3 shows the result of LOF technique: only three samples are exactly classified as 

outliers but there are two samples that this method does not recognize as outliers. 

Finally, in figure 4, is shown the result using fuzzy logic method. It clearly appears that all 

outliers have been correctly recognized. 
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Fig. 2. First example using Grubbs method. 
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Fig. 3. First example using Local Outlier Factor method. 
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Fig. 4. First example using Fuzzy Logic method. 

Similar results are obtained in the second application. 

The second examined variable is referred to a sensor detection made within the process. In 
this case, the samples considered outliers are 3,5,7,40 and 60 referring respectively to 
following values: 
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0.4730 0.4779 0.4779 0.4680 1.9217. 
The Grubbs test does not detect any outliers. The result is shown clearly in figure 5. 
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Fig. 5. Second example using Grubbs method. 

The Local Outlier Factor technique detects four outliers but only one is really an outlier. The 
result is shown in figure 6. 
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Fig. 6. Second example using Local Outlier Factor method. 
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The proposed method, as show figure 7, recognize all the outliers which are present in data. 
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Fig. 7. Second example using Fuzzy Logic method. 

6. Conclusions and future work 

A description of traditional approaches and of the most widely used methods within each 
category has been provided. As standard outlier detection methods fail to detect outliers in 
industrial data, the use of artificial intelligence techniques has also been proposed, because it 
presents the advantage of requiring poor or no a priori assumption on the considered data. 
A procedure for outlier detection in a database has been proposed which exploits a Fuzzy 
Inference System in order to evaluate four features for a pattern that characterize its location 
within the database. The system has been tested on a real industrial application, where 
outliers can provide indications on malfunctionings or anomalous process conditions. The 
presented results clearly demonstrate that the Fuzzy Logic-based method outperforms the 
most widely adopted the traditional methods. 

Future work on the FIS-based outliers detection strategy will concern the algorithm 
optimization in order to improve its efficiency and its on-line implementation. Moreover 
further tests will be performed on different applications. 
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1. Introduction 

It is well known that linear controllers can exhibit serious performance limitations when 
applied to nonlinear systems since nominal linear models used during design cannot 
represent the nonlinear plant in its whole operating range (Arslan et al., 2004). For this 
reason, several researches has been proposed new techniques in order to supply a solution 
for this problem. The main alternative technique, proposed by academy, to resolve the 
referred problem is known as multi-model approach. The basic idea of multi-model 
approach consists in decompose the system's operating range into a number of operating 
regimes that completely cover the chosen trajectory as showed in (Foss et al., 1995). There 
are, basically, two approaches for multi-model. The first one consists of to design a set of 
suitable controllers (one for each operating regime) and to calculate weighting factors to 
them as showed in (Arslan et al., 2004) and (Cavalcanti et al., 2007a). The global control 
signal is a weighting sum of the contributions of each controller. The second one consists of 
to build a global model as a weighting sum of each local model as showed in (Foss et al., 
1995) and (Cavalcanti et al., 2007b). In both cases, a way to measure distances between 
models is defined. Multivariable Model Predictive Control (MMPC) has been presented in 
this chapter. MPC is the an of the most important control technique used in industry. 
Multivariable Bilinear Generalized Predictive Control (MBGPC) is formulated and, its 
alternative solution, Multivariable Bilinear Generalized Predictive Control with Iterative 
Compensation (MBGPCIC) is presented. This chapter shows either proposed metrics in 
order to build multi-model based controllers (based in MBGPC and MBGPCIC) and 
presents simulation results applied in distillation columns. 

2. Multivariable Bilinear Generalized Predictive Control (MBGPC) 

MBGPC is a MPC technique based in the minimization of a objective function. This objective 
function considers the predicted output of a system. The prediction is obtained by a 
mathematical model of this system. 
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Considering a multi-input, multi-output (MIMO) system, as showed in Fig.l, with p-inputs 
and ^-outputs, being y the system's output and u the system's input. 



Ui 

u 2 



Up 



Multivariable 
System 



-►yi 
->y2 



-►y q 



Fig. 1. Block diagram, Multivariable system 

Considering that this system is described by the following matrix polynomial expression, 
showed in (Fontes, 2002): 

A( q - , )A^ q - , )y(k) = B(q- , )A p (q- , )u(k-\) + 

D e (</-' )D[u(k - \)]D d (q~ ] )A f (q~ x )y(k - 1) + (1) 

C(q l )e(k) 

where y(k) e R q is the process output vector, u(k) e R p is the process input vector and 

e(k) e R q is the Gaussian white noise with zero mean and covariance diag(cr 2 ) . The 

matrices A(q _1 ) , B(cf x ) and C(q~ 1 ) are polynomials matrices in shift operator q~ l and are 
defined by: 

A(q- 1 ) = I fXq +A l q- 1 +': + A M q- (2) 

B(q- 1 ) = B +B l q- 1 +... + B nb q- b (3) 

C(q- i ) = I pxp +C l q- l +-.. + C nc q- nc (4) 

D d (q l ) = D d0 +D d!l q- 1 + :. + D d>ndd q- nd < (5) 

DXq l ) = D eQ+ D eX q l + ^ + D end q^ (6) 

where A(q~ l ) e R qxq , B(q- l )^R qxp , C(q l )^R qM , D e (q~ l ) £ R qxp and D d (q l ) t R pxq . 
The matrix D[i/(£ - 1)] is defined as: 

Dfuik-l^dia^ik-l) u 2 (k-l) ••• w,(£-l)J (7) 

The nonlinear model presented in (1) is quasi-linearized to be used in MBGPC. The 
multivariable quasilinear multi-model must be obtained by rewriting the expression (1) of 
the following form: 

A(q-',u)A q (q-')y(k) = B(q-')A p (q-')u(k-l) + C(q')e(k) (8) 
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where 

A(q-\u) = A(q- 1 )-q- l D e (q- l )D[u(k-l)]D d (q- 1 ) (9) 

The polynomial matrix A(q~\u) is calculated considering its parameters as constant in 
prediction horizon. The polynomial matrix A(q~\u) is considered diagonal in this work. 
The output prediction i-step ahead may be obtained of the expression (8), such that: 

A(q l , u)y(k + = B(q l )A p (q 1 )u(k + i - 1) + C(q l )e(k + i) (10) 

where A(q~\u) = A(q~ l ,u)A (q' 1 ) . In this case, the polynomial matrix Cfq- 1 ) is equal to 

Ipxp due the fact that the noise be supposed white. Considering the following Diophantine 
equation: 

I pxp =E i (q-\u)A(q-\u) + q- i F i (q-\u) (11) 

where 

E i (q l , u) = E io (u) + • • • + E t ._ x (u)q- (i - l) (12) 

F i (q~\u) = F i Xu) + --' + F ina (u)q~ na (13) 

Pre-multiplying (10) for E i (q~ l , u) we obtain: 

E. (q " ! , u)A(q " ! , u) y(k + 1) = E. (q " ! , u)B(q _1 ) A p (q " ! )u(k + i - 1) + E. (q " ! , u)e(k + /) (14) 
Rewriting (9) of the following form: 

E i (q-\u)A(q-\u) = I pxp -q- i F(q-\u) (15) 

Substituting (15) in (14) we obtain: 

y(k + i) = F t (q l , u)y(k) + E. (q l , u)e(k + /) + E. (q l , u)B(q l )A p (q l )u(k + i - 1) (16) 

As the degree of E.(q~ l ,u) is i — \, then the sub-optimal prediction of y(k + i) is: 

y(k + i) = F i (q~\u)y(k) + E i (q~\u)B(q- l )A p (q- l )u(k + i-l) (17) 

In order to separate past and future values, we make: 

E i {q\u)B{q-')=H i {q\u)+q'KXq\u) (18) 

As the degree of H. (q~\u) is less than i - 1 , the predictor may be written as: 

y(k + i) = F (</" , u)y(k) + H. (q- 1 ,u)A p (q x )u(k - 1) + H t (q x , u)A p (q~' )u(k + i - 1) (19) 
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The last term of (19) considers the future inputs (forced response) and the two first terms 
consider only past inputs (free response). So, we define: 



y(k + = HXq~\u)k p {q- l )u{k + /-!) + Y u 



where 



Y u =FXq-\u)y{k) + H ipa {q\u)^{q x )u{k-l) 
The expression (21) is the free response. The objective function is given by: 

NY NU 

J = Yjr(k + i) - y(k + i)f R + JjMk + i - % 



(20) 



(21) 



(22) 



where A^ is minimum prediction horizon, NY is prediction horizon, NU is the control 

horizon, R and Q are weighting matrices of error signal and control effort in instant k in 

the chosen trajectory, respectively, y(k + i) is the sub-optimum i-step ahead predicted 

output, r(k + i) is the future reference trajectory. The control effort is obtained, without 

constraints, by the minimization of the objective function (22). 
Consider the predictions set: 



where 



y Nly = H N^ P U NU +Y lN x y 

y Ni =\y(k+N x ) yQc+ty+l) - yQc+NTft 



H K 



\U NU 



J-f T~f T~f 

TT TT TT 

N, N x -\ '"' N x +\-NU 



TT TT TT 

rL NY-1 £J NY -2 " " ' £J NY-NU 



A p (q->)u(k) 
A p (q-')u(k + l) 



A (q- v )u{k + NU-\) 



(23) 



(24) 



(25) 



(26) 



ymu 



(27) 
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The objective function (22) may be rewritten of the following form: 

J=(H Ni A p U m +y lNiy ) T R(H N ^ p U NU +y IK )+Alf NV QA p U NU (28) 
where 

R=diag[R„-,R vm ] (29) 

Q=diag[Q 1 ,-,Q pxNU ] (30) 

This minimization is obtained by the calculation of its gradient (making it equals zero), of 
the following form: 

dJ (31) 



dA U, 
The minimization of (28) produces the following control law: 

AU NU = {H T N H^ + QY H T N R(r - y^ ) (32) 

Because of the receding control horizon, only the first p rows of (32) are computed. 

3. Multivariate Bilinear Generalized Predictive Control With Iterative 
Compensation (MBGPCIC) 

3.1 Motivation 

The quasi-linearization presented in (8) produces a prediction error that degrades the 
controller performance. This prediction error increases with the prediction horizon. In order 
to solve this problem, several algorithms has been proposed (Fontes et al., 2002), (Fontes et 
al. 2004), (Fontes and Angelo, 2006) and (Fontes and Laurandi, 2006). This section presents 
the multivariable case showed in (Fontes and Laurandi, 2006). 

The basic quasi-linear algorithm, presented by (Goodhart, 1994), calculates the output 
prediction z-step ahead considering the terms A (q~\u) with j = l,--,na depending only of 

known values of the input (until k-1 step). The approximation of this approach generates a 
prediction error that increases with the prediction horizon and degrades the controller 
performance. 

3.2 The basic idea of iterative compensation algorithm 

The idea of the iterative compensation algorithm consists of consider the effort control 
sequence (obtained by the classic quasi-linear algorithm) to correct the parameters of 
A(q~\u) . Considering the following sequence of effort control, calculated in k step: 

AU NU =[^(k) ••• tu,{k) ••• Au^k+NU) ••• *u p (k+NU)] (33) 

The classic quasi-linear algorithm, considering (30), calculates the sequence of future 
control efforts, that is given by: 
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U NU =\ Ui {k) ... u p (k) .- i^+JW) .- Mp (£+7W)] r (34) 

where: 

Uj(k + = ufo - 1) + ^ AUj(k + t),j = l,'~,p (35) 

t=0 

The iterative compensation algorithm uses the values of (34) to correct the polynomial 
matrix (9). This correction will produces a new prediction and a new sequence of control 
efforts. This process is repeated until a stop criterion be achieved. It is important to 
remember that this algorithm considers the receding horizon too. When the algorithm 
converges, only the first p rows of (34) are sent to the process. 

3.3 Convergence and stop criterion 

The stop criterion of the algorithm is based in the norm variation of the vector A U NU . The 

procedure of correction of A{q~\u) is repeated until that the norm of the variation 
calculated in the iteration r be less than a tolerance value £ established: 

^KU NU , r -A p U NUr J(A p U N ^ ~A p U N u, r J < £ (36) 

It is important to remember that this algorithm may not to converge. In this case, another 
stop criterion must be established. The second stop criterion is based in the maximum 
number of iterations of the algorithm: 

r < N max ( 37 ) 

where 7V max is the maximum number of iterations of the algorithm. If the algorithm stops for 
the N max criterion, the control effort sent to the process is the control effort calculated by the 
classic quasi-linear algorithm. 

4. Multi-models approaches for MPC 

Considering a bilinear multivariable model showed in (1) that describes the system's 
behavior in a small region. This structure is valid around its operating regime and more or 
less invalid outside this regime. Considering yet that the process has been decomposed into 
NOR operating regimes, the first step to develop a multi-model structure is to identify NOR 
local models (in this case bilinear), of the following form: 

A U) (<?-' ) A, (<?-' )y{k) = B u) (q-')A p (q- , )u(k-l) + 

D HJ) (q- 1 )D[u(k - \)]D d{J) (q-' )A, (</" )y(k - 1) + (38) 

c , (<?->(£) 

where y =!,■■■, NOR. 
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4.1 Building a global model 

The first multi-model approach consists in to obtain a global model from the local bilinear 
models showed in (38). To build the global model, we must consider that there is a validity 
function 8 j (k) that is designed such that its value is close to one for operating points where 
the local model structure is a good description of the system and close to zero otherwise, in 
instant k. In this case, each polynomial matrix P (k) (q~ l ) of (1) for each instant k would be 
calculated of the following form: 

NOR 

^ i) (?-') = £^fe- , )w w (38) 

where P (q~') the global built polynomial matrix of the bilinear model, P,Aq' 1 ) is the 
polynomial matrix of the;'* bilinear model, and: 

S(k) 
Wy^TST 12 -; J = h-;NOR (39) 

£$(*) 

t=\ 

where w. k is a weighting factor to the j th bilinear model in instant k. Theses approaches has 
the following defined property: 

NOR 

£w,,=l (40) 

7=1 

4.2 Building a global controller 

The second multi-model approach consists of to build a global controller from a set of 
controllers. In this case, one controller is designed to each operation point. A validity 
function S.(k) is designed too, in order to evaluate what controller must have a greater 

weighting factor. Considering the control effort u i (k) with j = 1, • • • , NOR and i = !,•••, p 

of each controller (one for each operation point), the control effort sent to the process is 
given by: 

NOR 

M( (£)=2>. tMiJ w (4i) 

7=1 

where w. k is calculated as showed in (39). 

4.3 Metric based in norms 

The validity function is usually called by metric. In multivariable case, in a process with p- 

inputs and q-outputs, the output is y(k) e R q and the input is u(k) e R p . In a known 

trajectory of process output, the distance from the first operation point to the last operation 
point is given by: 
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d i, N0 R =\\y N oR-yi\\ 2 ( 42 ) 

To measure the distance from the current operation point to the operation point of j th 
designed controller, we can use the expression: 

<?. = „ dhN0R |, ; i = l,-,NOR (43) 

\\y( k *)-yl 

To this metric, only monotonic (increasing or decreasing) trajectories must being considered. 

4.4 Metric based in phase margin 

Phase margin has been chosen as measurement parameter in order to quantitatively 
estimate the distance between two different models. 

One of the most important techniques for measure the robustness of a dynamic system is the 
margin phase technique. In general terms, in a linear time-invariant system, phase margin is 
maximum phase angle that can be added, such as this system not becomes instable. In this 
chapter, a multivariable bilinear model is obtained for each operating regime, and this 
model is quasi-linearized. Each obtained model has a phase margin value (or multivariable 
equivalent). An interpolated model is calculated to obtain the current valid model (that 
depends of the current operation point). The difference between the interpolated model and 
models of the chosen operating regimes is calculated. This difference is considered to the 
computation of a set of weighting factors to the controllers. An equivalent method to the 
multivariable margin phase calculus is showed. 
The model showed in (8) is time-step quasi-linear (linear at each time instant). We obtain a 

transfer function in z _1 operator, pre-multiplying (8) for A~ x (z _1 ,u) of the following form: 

y(z- l ) = A w \z-\u)B w (z- i )z- l u(z- 1 ) (44) 

where A (k) (z~\u) and B (k) (z~ l ) are interpolated polynomial matrices in instant k. From 
(44), the matrix transfer function for the interpolated model in instant k is given by: 

G ik) ( z ~ 1 > u ) = A ~\v _1 > u ) B (k ) ( z_1 ) z_1 ( 45 ) 

In this approach, we consider the pxq transfer functions from (45), and calculate 
individually the phase margin from each one transfer function. 

For each operating regime, the minor margin phase is chosen. Considering that MPM (J) 
represents the minor phase margin of the matrix transfer function in the j th operating regime 
and that max(MPM (i) ) is the maximum of phase margin for j = 1, • • • , NOR and 

mm(MPM (j) ) is the minimum phase margin for j = 1, • • • , NOR . 

Considering yet that G (k) (z~\u) is an interpolated transfer function given in k instant, and 

MPM {k) is its minor phase margin, the distance factor to the designed controller in the ] th 
operating regime, in instant k is given by: 
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1- 



\MPM {k) - MPM { 



max(MPM 0) ) - mm(MPM U) ) 



(46) 



It is important to observe that the interpolated model's parameter (in this case, obtained by 
cubic spline interpolation method) is always in the bounds of the estimated model's 
parameter (it is guaranteed by the algorithm). The algorithm guarantees also that, if a 
calculated phase margin is out of bounds mm(MPM {l) ) and max(MPM (0 ) , the designed 
controllers of these bounds have weighting factor equals 1 (maximum). 



5. Applications of multi-model controllers based in proposed metrics 

This section shows some applications of the combinations between: 

• approaches showed in sections 4.1 and 4.2; 

• controllers showed in sections 2 and 3, and; 

• metrics showed in sections 4.3 and 4.4. 

The application showed in this chapter is based in a simulated debutanizer distillation 
column. Debutanizer distillation column is usually used to remove the light components 
from the gasoline stream to produce Liquefied Petroleum Gas (LPG). The most common 
control strategy is to manipulate the reflux flow rate and the temperature in column's 
bottom and, to control the concentrations of any product in butanes stream and in C5+ 
stream as showed in (Almeida, et al., 2000). The chosen process variables are: concentration 
of i-pentane in butanes stream (yi) and concentration of i-butene in C5+ stream (1/2). The 
studied column is simulated in Hysys software and is showed in Figure 2. 
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Fig. 2. Distillation Column simulated in Hysys Software 
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The reflux flow rate (m) is manipulated through the FIC-100 controller and the temperature 
of column's bottom (w 2 ) is manipulated through the TIC-100 controller. The reflux flow rate 
is measured in m 3 /h and the temperature of column's bottom is measured in °C. 
In the applications showed in this chapter, three operation points were chosen, as showed in 
Table 1. The identified bilinear models were obtained using the multivariable recursive least 
squares algorithm and the model's structure has been chosen by using the Akaike criterion. 
In all points, the chosen sample rate is 4 minutes. Only monotonic trajectories are being 
considered. The trajectory of y x is monotonically increasing and the trajectory of y 2 is 
monotonically decreasing. 



Operation 
Point 


Input 


Output 
(Mass Fractions) 


1 


ui = 40 m3/h 


Yi = 0.014413 


u 2 = 147 °C 


Y 2 = 0.001339 


2 


ui = 37 m 3 /h 


Yt = 0.017581 


u 2 = 147.5 °C 


y 2 = 0.001161 


3 


ui = 34 m 3 /h 


Yt = 0.021994 


u 2 = 148 °C 


Y 2 = 0.001004 



Table 1. Three operation points chosen in distillation column. 

In order to quantitatively asses the performance of multi-model quasi-linear GPC, some 
indices like showed in (Goodhart, et al., 1994) are calculated. Theses indices may be 
extended to multivariable case, of the following form: 



s x . =^\u.(k)\/N 



(47) 



where i = l,~-,p and TV is the amount of control effort applied in the process to achieve 

the desired response. The index showed in (47) is the account of total control effort to 
achieve a given response. The variance of controlled actuators is: 



The deviation of the process of integral of absolute error (IAE) is: 

Hj = I\ r M)-y\ IN 

where j = l,---,q . The overall measure of effectiveness is defined as: 



(48) 



(49) 



S i = H ( a i £ U + Pi e u ) + Pi £ u 



(50) 



The factors a t , /?. and p are weightings chosen to reflect the actual financial cost of 
energy usage, actuator wear and product quality, respectively. In this case, we consider 
a i =0.1, P i =0.15 and p =0.5 because we have established as priority the product quality. 
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5.1 Application 1 - Controller based in global model, norm-2 metric and MBGPC 

In this simulation, the process is in the 3 rd operating regime and a deviation in reference is 
applied in the proposed controller. With this reference deviation, the process will come to 
close to the 1 st operating regime. The proposed quasi-linear multi-model is compared with 
quasi-linear single-model (using the model of the 3 rd operating regime). Figures 3 and 4 
show the process's output and Figures 4 and 6 show the control effort. 
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Fig. 3. Process Output 1. Comparison between single-model and multi-model approach 
(application 1). 
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Fig. 4. Process Output 2. Comparison between single-model and multi-model approach 
(application!). 
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Fig. 5. Reflux Flow rate. Comparison between single-model and multi-model approach 
(application!). 
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Multi-Model 
- Single-Model 



Fig. 6. Temperature in column's bottom. Comparison between single-model and multi- 
model approach (application 1). 

Figures 3, 4, 5 and 6 shows the better performance of multi-model approach, so much of 
point view of the process response as of the control effort. 



5.2 Application 2 - Controller based in global controller, norm-2 metric and MBGPC 

In this simulation, the same reference deviation as in section 5.1 is applied. The proposed 
quasi-linear multi-model is compared with quasi-linear single-model (using the model of 
the 3 rd operating regime). Figures 7 and 8 show the process's output and Figures 9 and 10 
show the control effort. 
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Fig. 7. Process Output 1. Comparison between single-model and multi-model approach 
(application 2). 
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Fig. 8. Process Output 2. Comparison between single-model and multi-model approach 
(application!). 
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Fig. 9. Reflux Flow rate. Comparison between single-model and multi-model approach 
(application 2). 
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Fig. 10. Temperature in column's bottom. Comparison between single-model and multi- 
model approach (application 2). 

5.3 Application 3 - controller based in global controller, phase margin metric and 
MBGPC 

In this simulation, the same reference deviation as in section 5.1 and 5.2 is applied. The 
proposed quasi-linear multi-model is compared with quasi-linear single-model (using the 
model of the 3 rd operating regime). Figures 11 and 12 show the process's output and Figures 
13 and 14 show the control effort. 
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Fig. 11. Process Output 1. Comparison between single-model and multi-model approach 
(application 3). 
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Fig. 12. Process Output 2. Comparison between single-model and multi-model approach 
(application3). 
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Fig. 13. Reflux Flow rate. Comparison between single-model and multi-model approach 
(application3). 




Fig. 14. Temperature in column's bottom. Comparison between single-model and multi- 
model approach (application 2). 



5.4 Qualitative comparison between applications 1,2 and 3 

This section shows the qualitative comparison between application 1,2 and 3. The 
comparison is based in the indices showed in (47), (48), (49) and (50) with N=100. 
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The table 2 shows, in all the cases, that multi-model approach has better performance in 
relation of single-model approach in terms of less energy usage, less actuator wear and 
better product quality. Application 2 has presented better performance in relation of the 
other multi-model approaches. 
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Single 


40.47 


2.61 


287.46 
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Single 
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Application 1 
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0.31 
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Application 1 


146.88 


0.36 
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Application 2 


38.38 


0.32 


248.41 
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Application 2 


146.94 


0.29 
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70.36 
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Application 3 


38.55 


0.33 


253.41 


145.36 
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Application 3 


147.01 


0.31 


113.71 


75.51 



Table 2. Qualitative comparison between applications 1,2 and 3. 

5.5 Application 4 - controller based in global controller, norm-2 metric and MBGPCIC 

In this simulation, the process is the following operation point: m = 31 m 3 /h, U2 = 148.5 °C, 
yl=0.028125 e y2=0.000874. The reference deviation 0.01371 and 0.000465 are used in this 
application. This approach has been compared with application 2. Figures 15 and 16 show 
the process's output and Figures 17 and 18 show the control effort. 
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Fig. 15. Process Output 1. Comparison between MBGPC and multi-model approach and 
MBGPCIC multi-model approach (application 4). 
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Fig. 16. Process Output 2. Comparison between MBGPC and multi-model approach and 
MBGPCIC multi-model approach (application 4). 




Fig. 17. Reflux Flow rate. Comparison between MBGPC and multi-model approach and 
MBGPCIC multi-model approach (application 4). 
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Fig. 18. Temperature in column's bottom. Comparison between MBGPC and multi-model 
approach and MBGPCIC multi-model approach (application 4). 

Figures 15, 16, 17 and 18 show the better performance of MBGPCIC multi-model approach, 
so much of point view of the process response as of the control effort. The justification to 
this significant improvement of MBGPCIC multi-model approach in relation to MBGPC 
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multi-model approach is the wide variation in control effort. So, the iterative compensation 

procedure founds more space to minimize the prediction error. 

Due too the large operating range, this approach has not been compared with single-model 

approach, because its performance was very poor. 

Table 3 shows the performance indices of this comparison for N=400. 



VO 


Approach 


*i 


s 2 


*3 


s 
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MBGPCIC Approach 


50.47 


6.61 


1219.90 


632.02 
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MBGPCIC Approach 


40.71 


5.78 


1151.12 
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MBGPC Approach 


148.27 


8.00 


952.00 


498.07 


2 


MBGPC Approach 


147.21 


1.73 


260.98 


150.41 



Table 3. Qualitative comparison between MBGPC and multi-model approach and 
MBGPCIC multi-model approach (application 4). 

6. Conclusion 

This chapter showed the importance and the relevance of multi-model approaches. 
Several researches has been proposed in order to solve design problems in process that 
operates in a large range (like batch processes). Some proposals of multi-model have been 
presented in this chapter and its comparison with classic approaches. All multi-model 
approaches presented better performance when compared to single-model approach. The 
next step of this research is to adjust these approaches to a robust and stable algorithm of 
multi-model. 



7. References 

Almeida, E.; Rodrigues, M.A. & Odloak, D. (2000). Robust Predictive Control of a Gasoline 

Debutanizer Column. Brazilian Journal of Chemical Engineering, vol. 17, pp. 11, Sao 

Paulo. 
Arslan, E.; Qmiurdan, M. C; Palazoglu, A. & Arkun, Y (2004). Multi-Model Control of 

Nonlinear Systems Using Closed-Loop Gap Metric. Proceedings of the 2004 American 

Control Conference, Vol.3, pp. 2374-2378, Boston. 
Cavalcanti, A. L. O; Fontes, A. B. & Maitelli, A. L. (2007a). Generalized Predictive 

Control Based in Multivariable Bilinear Multimodel. Proceedings of 8th 

International IF AC Symposium on Dynamics and Control of Process Systems, pp. 91- 

96, Cancun. 
Cavalcanti, A. L. O; Fontes, A. B. & Maitelli, A. L. (2007b). A Multi-Model Approach For 

Bilinear Generalized Predictive Control. Proceedings of 4th International Conference on 

Informatics in Control, Automation and Robotics, pp. 289-295, Angers. 
Cavalcanti, A. L. O; Fontes, A. B. & Maitelli, A. L. (2004). Bilinear generalized predictive 

control: an adaptive approach. Proceedings of 5th Asian Control Conference, Vol. 3, pp. 

1781-1785, Melbourne. 



300 Advances in Robotics, Automation and Control 

Fontes, A.; Maitelli, A. L. & Salasar, A. O (2002). A New Bilinear Generalized Predictive 
Control Approach: Algorithm and Results. Proceedings of 15th Triennial IP AC World 
Congress, pp. 235-239, Barcelona. 

Fontes A. B. (2002). Desenvolvimento e Avaliacao de Controladores Preditivos Baseados em 
Modelos Bilineares, PhD Thesis, UFRN, Natal. 

Fontes, A. & Angelo, E. (2006). Controle Preditivo Generalizado Bilinear Aplicado A Uma 
Coluna de Butadieno 1,3: A Compensacao Iterativa, uma nova abordagem. 
Proceedings of 16th Brazilian Congress of Automatic, pp. 863-868, Salvador. 

Fontes, A. & Laurandi, S. (2006). Controlador Preditivo Generalizado Bilinear Multivariavel 
com Compensacao Iterativa, Uma Nova Abordagem. Proceedings of 12th Latin- 
American Congress on Automatic Control, pp. 133-138, Salvador. 

Foss, B.A.; Johansen, T.A. & Sorensen, A.V (1995). Nonlinear Predictive Control Using Local 
Models - Applied to a Batch Fermentation Process. Control Eng. Practice, pp. 389- 
396. 

Goodhart, S. G.; Burnham, K. J. & James, D.J.G. (1994). Bilinear Self -tuning Control of a high 
temperature Heat Treatment Plant. IEEE Control Theory Applications, Vol. 141, n° 1, 
pp. 779-783. 



16 



The Holonic Production Unit: an Approach for 
an Architecture of Embedded Production 

Process 

Edgar Chacon, Isabel Besembel, Dulce M. Rivero and Juan Cardillo 

Universidad de Los Andes, Facultad de Ingenieria Escuela de Ingenieria de Sistemas, 
Av. Alberto Carnevalli, Complejo la Hechicera, Edf. B. piso 2, Merida- 

Venezuela 

1. Introduction 

The holonic system paradigm derives from the word "Holon" that was coined by Arthur 
Koestler (Kloester, 1967), and it comes from the composition of the Greek word "holos" that 
means "the whole" and the suffix "-on" that indicates part. According to Koestler, every 
holon has the same structure, that is stable, coherent and its components has the same 
structure; the organization is invariant, the holon can be part of a bigger holon and it has 
parts that also are holons. This result was obtained by Koestler in the search of auto- 
organizational models for biological systems, where he identified structural patterns that 
form hierarchical embedded structures that are copies of their. This embedded hierarchy 
was named holarchy; this term reflects the holon trend to act as autonomous entities that 
cooperate in order to achieve an hierarchical system auto-organized similar to 
cells/ weave/ organ hierarchical systems in biology (Christense, 1994). In the holarchy, 
holons, independently of its aggregation level, have a behaviour as a whole and as parts that 
cooperates to reach an established goal. Inside the holarchy, holons can simultaneously 
belong to different groups, having precise rules that describe its behaviour. Internally, its 
own rules define the holon as an individual entity, with a structure and a functional model 
and its properties. In a holarchy, the autonomy duality, as a principal contradictory 
opposition, it is balanced by the knowledge model, which defines the functionality of the 
system as a composition of semiautonomous holons. 

The Holonic Enterprise (HE) (McHugh et al, 1995) emerges based on the concepts given in 
the upper paragraph in order to satisfy the enterprise paradigm of having reconfigurable, 
open, and flexible production models, which are able to follows the dynamic of the market 
in a global economic-network. The HE has strategies and relationships that evolve with the 
time according to the business environment dynamics. 

Nowadays, formal enterprises look for the accomplishment of the openness paradigm, able 
to reconfiguration, to be flexible by the creation of autonomous production units able to 
cooperate or dynamic enterprise networks build to accomplish a specific goal. In this way, 
an ideal organization or a virtual enterprise is a set of different production units that are 
geographically distributed simulating to be a unique organization with a specific 
localization. For a virtual enterprise, it is necessary the existence of a technological 
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communication infrastructure that communicate all of the units that belongs to the virtual 
organization (Ulieru et al, 2002). 

Actually, there are several models based on the holonic approach for the manufacturing 
systems and the architectures that support them (Kloester, 1967), (Chistense, 1994), (Brusel 
et al, 1998), (Hannover, 2000), (Brussel et al, 1998), (Wyns, 1999). In (Chacon et al, 2007), it is 
shown a schema for a holonic production processes that incorporate complex - precise 
models for the whole system. 

In this work, we propose a reference model for the production processes in virtual or 
networked enterprises, which is seen as the cooperating composition of Holonic Production 
Units. The relationship among the different plants of the production process is derived from 
the business model, the value chain, and the production flows (Cardillo et al, 2005). These 
Holonic Production Units (HPU) are described in terms of their components, where each 
one has the holon characteristics. The invariant composition procedure, without lost 
generality, is given by the recursive composition of the holons as was mentioned in previous 
holon definition. 

The basic holon is composed of non-autonomous equipment (resources) that performs direct 
product transformation, storage and transportation tasks, direct control and regulation 
systems, the supervision system, and the management system for the holon. Resources can 
be controlled by an exclusive controller for each equipment and the set of resources and 
controllers shapes the controlled basic holon. This basic holon must have the capability to 
supervise and coordinate its own set of controllers; in this case we call an autonomous 
supervised basic holon. If the basic holon must be able to negotiate production goals and to 
synchronize the production scheduling with other basic holon to ensure a production goal; 
in this case we have the expected behaviour for the holon. In figure 1 it is shown the 
functional structure and decision elements for a basic holon. 
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Fig 1. The basic holon 

A set of basic holons or a basic holon that attain a goal is defined as the Production Unit 
Holon (PUH). This PUH keeps the holon structure and the control and regulation 
functionalities. Then, it must have control mechanisms, a supervision mechanism and 
negotiation / coordination mechanisms with other PUH at the same level. 
In section 2 it is presented how to model formal enterprises from a holonic point of view. 
Section 3 describes the Holonic Production Unit that is oriented to the achievement of a goal 
that is derived from the value chain. In section 4, we present the holonic enterprise like a 
composition of Holonic Production Units. Conclusions and future works are given in section 5. 
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2. Modelling formal enterprises. 

As has been presented in the before section, a production process must be modelled in the 
same way, independently if the kind of enterprise (formal or virtual) that will be organized 
to accomplish a production goal. The production process depends only of the value chain 
and the production flows. The production process necessary to obtain the production goal 
follows a model where each element of the value chain represents a stage of the production 
process. Each stage adds value to a product by transformation, transport or storage of that 
product across the value chain (Chacon et al, 2002) as it is shown in figure 2. 
The production flow can be defined as the set of transformation stages determined by the 
production method using a set of resources specific for each stage (recipe) until to obtain the 
final product. Each stage must ensure the quality of the process and the products that are 
obtained. 




Fig. 2. The Value Chain 

2.1 Description of a production unit in production process 

Each stage for the value chain (raw material input / processing or transformation / storage) 
of the production flow is seen as performed by a Production Unit. Then, a production 
process is an aggregation of cooperating production units. The characterization of each 
production unit depends on how the resources evolve on the production process at the 
production unit. The resources dynamic can be: Continuous, Discrete or Event Dynamic 
(manufacturing), or Hybrid (for example in batch processes). Additionally, each PU 
performs operations that follow a recipe for the equipment that belongs to the production 
unit. 

It is possible to find common elements that allow describing the PU in a generic way. Then 
a Production Unit must have: 

• A process to obtain raw material 

• A transformation process 

• A storage process for final or intermediate products 

At begin, production resources (equipment, HR, and raw material) are managed internally 
or requested to other PU by means of UP coordinator. Then resources are localized and 
translated from a UP to the UP that requires resources and intermediate/ final products are 
translated from a UP to the client that can be extern to the enterprise or other UP. Thus, the 
resources localization and translation operation is responsible to guarantee the resources for 
a production process corresponding to a production method. The selection of the production 
method depends on the quality and quantity of raw material, capability and available 
capacity of the equipment. When the production process ends, resources are liberated and 
the obtained products are temporally stored, until other PU or final client request for the 
product. 
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In figure 3, we show the structural model of a PU with all its elements. This model includes 
the control, supervisory, and coordination/ management system for each stage of the value 
chain. A PU drives the production methods and the internal configuration of the internal 
resources in order to obtain the production goal. 

Using the above UP definition, we establish the basis to obtain the information that allow to 
fix a production schedule and to determine the state across the production process; Key 
Production Indicators: production quality, production capacity, expected production, etc., all 
of them necessaries to negotiate with other PU a production goal. A production agreement 
results from the production capacity and the existence actual or future of raw material. 
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Fig. 3. Production model for a Production Unit 

The structural model for the PU is obtained using UML (Cardillo et al, 2005), (Eriksson et al, 
2004), (Muller, 1997), (Jacobson, www), and (WBF, www), where boxes represent classes 
and lines among the boxes describe relations among the classes. Lines describe three kinds 
of relationships: generalizations / specialization by arrows, associations by plain lines and 
compositions by lines with a rhombus in the end. 

For the model shown in figure 4, we can see different entities that are components for a PU 
and its relations. Specially, it is shown an association, which record all different resource 
configurations, the production process, and control and supervision mechanisms for a 
particular production. 

Also, it shows a classification for the resources used in production, that is employed by the 
planning function belong to the PU. The whole model has a specialization for several classes 
that they are not shown in the schema; each resource class has a behaviour that is described 
by means of a Discrete Event Dynamical System. The dynamics describes rules and 
operations for each class and the composition of dynamics define the behaviour of the PU. 
Using the UML model shown in figure 4, a figure 5 represents the embedded procedure of 
making decision process for each stage in the value chain that allows considering each stage 
as an autonomous Production Unit. 

The embedded model corresponds to the decomposition of the making decision process, 
which describes control procedures, supervision/ coordination schemas. This reference 
model establishes three levels: the floor level where we can found the physical process 
(Transformation) and its control loops (observers, controllers, and transducers). All of them 
installed over the Control, Information, and Communications Technology. 
In this way, the physical process is seen as the system to be monitored, supervised, and 
managed by an upper level that is the supervision level. Without lost of generality, a PU 
abstraction allows us to see as a physical process that can be part of a new stage in the value 
chain. This stage receives raw material, handle it, perform transformations by means of its 
own resources and generates new products for the next stage on the value chain or for the 
final client. 
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Fig. 4. The UML model for a PU 
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Fig. 5. Embedded Model of the Production Process 



3. The holonic production unit 

A holon for manufacturing enterprises is defined as a cooperative and autonomous 
construct able to transform, transport, store and/ or validate information and/ or physical 
products, [8, 12, 17, 18, 21]. A holon has the autonomy to create and control the execution of 
its own plans; it can cooperate with other holons to develop a viable plan in order to reach 
the global goal for the system. The cooperation among the holons is accomplished by means 
of a holarchy that evolves from the same organization. We have a holonic system. 
For a holonic production system, the goal is to obtain all the facets for the decision and 
control procedures from the establishment of a production plan for the whole PU until the 
control/ regulation at the floor level. The integrated automation is based upon a global 
vision of the production process, where each element that make part of the system must be 
take into account in order to control, supervise, and manage the production; see figure 6. 
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The automation schema is given by the construction of models that represents the PU in its 
structural and behavioural aspects. Control schemas ensure that the PU behaviour can be 
guaranteed inside a set of restrictions. The knowledge of the state of the physical system 
allows evaluating which will be the control actions that are viable in order to attain the 
desired state. At the system, the control is restraint by the accepted goal for the PU. The 
accepted goal is evaluated and negotiated taking into account the capabilities and 
availability of the PU. 



Control ) Management ) Maintenance 



Process 



Fig. 6. Integral Automation Model of Production Systems 

All the knowledge of the PU is internal to the PU, which has a behaviour that follows the 
physical/ chemical laws for an operation mode. The composition of the laws for each 
component gives the behaviour of the whole PU. This description is the base to establish the 
control and supervision methods that can be performed by humans or by the technological 
system. Each production resource has a proper dynamic; the changes across the time and 
the information recovered from the process determine the accomplishment of the 
production goal or the impossibility to attain the goal due to the failure of one or more 
resources. Those changes must be managed by the proper PU, in order to solve internally 
with its own resources by means of a new internal configuration, or renegotiate in case to be 
necessary and establish a new internal goal. 

To build a PU, it is necessary to fix implementation architecture for the PU. We propose to 
fix a structure composed by a "BODY" where the physical process (transformation 
/storage/ transport) are instantiated, and they are performed using the physical assets of the 
enterprise, such as: reactors, warehouses, pipelines, etc. A "HEAD" that instantiate all the 
decision process, which are based on the knowledge of the plant, and is constituted by 
human and machines. A "NECK" that is the interface between the BODY and the HEAD that 
supports de decision making system and store, transport information, data, and knowledge of 
the PU; the NECK is spread on the Information & Communication Technology. 
In figure 8, it is shown the correspondence between the HPU structure and the common 
technology found in production systems. Then, the body is the production process itself, the 
neck is composed by the networks, sensors, actuators, that allow obtain, transport data and 
by computers, industrial computers that process and store information/ knowledge. Head is 
constituted by the humans and algorithms that take decisions. 

In this way, we propose a recursive holonic structure that describes the desired composition 
for the Holonic Production Unit (HPU). The HPU is composed by one PU or by an 
aggregation of PU, which are holons. The Production Resource Holon, which be named 
Basic Holon results from the aggregation of basic process that are at this time controlled, 
supervised, coordinated, and managed. 
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BODY 



Fig. 7. Composition of a Holon in Production Process 

Each basic holon has as body a controlled process, that it is formed by all the control loops. 
Each holon has the structure shown in figure 9, where its intelligence allows attaining the 
desired state by the Controller. 




Fig. 8. Relationship between holon and the TIC infra-structure 



3.1. Holonic control loop 

In classical control, the basic functional unit is the control loop. If to the control loop we add 
as part of it, the fault detection capability and the capability of selection of new control 
strategies according to the operational condition, we talk about a Holonic Control Loop, the 
set of dynamical equations for different operational conditions must be part of the 
knowledge of the holon, also the controllers associated to each conditional operation. Neck 
for this holon is the set of elements that measure, adequate, transmit, and store physical 
signals from/ to the process. Head has the mechanisms that allows determine the 
operational conditions and perform the control algorithms. Those algorithms are derived 
from the physical models of the plant and the control mechanisms that were selected for 
each operational condition. 
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Fig. 9. Holonic Control Loop 

The self-diagnostic mechanism allows to measure efficiency, conditions of the equipment; to 
establish requirements for raw material and equipment in order to perform the negotiation 
and to manage the maintenance activities. Then the holon with control loop is an 
autonomous system where raw material is received and final products are generated 
according a negotiated goal; it fix itself set-points select control strategies and control 
parameters. See figure 9. 



3.2. Holon with process supervision 

As we know, a process can have several control loops. Each control loop corresponds to an 
element that must be coordinated with others in order to have a valid operation. The set of 
control loops corresponds to a controlled holon. The process management of the set of 
control loops corresponds to a supervisor/ coordinator. This making decision mechanism in 
classical supervision establishes set points, controller parameters or a control selection for a 
production method. If we add mechanisms to detect abnormal situations due to failures in 
control loops and incoherent states on the individual control loops for the global system, we 
have a supervised holon or holons with process supervision. 

Without losing generality, the Holonic Controlled Process is a process that has i models 
(maybe more than one for a controlled loop), where each model has a combination of m of 
the nominal values of operation that can be reached with a combination of j of the 
controllers' types which are adjusted under an criteria/judgement determined by the loop 
with n parameters as it can seen in figure 10. 

The Supervised Process Holon is conformed by a body that contains to the Holonic 
Controlled Process whose output is the state of the equipment and deviations of the consign, 
and whose input is the consign, type, and parameters of the controller to use, due to a set of 
tasks to complete that they have been evaluated and negotiated previously. 
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Fig. 10. Holonic Controlled Process 

Figure 11 presents the Holonic Supervised Process, the neck conformed by all of the tele- 
informatics architecture and applications, which allow to detect and to send events (they are 
able to capture, to try, to store, to adapt, continuous information in events and vice versa). 
The neck sends consign to the Holonic Controlled Process. From the method of production 
instanced and it detects the states generated by the Holonic Controlled Process. The head or 
fault tolerant supervisor, this conformed by the mechanism of taking of decisions capable of 
to establish by agreement/ negotiation the set of tasks to carry out for Holonic Controlled 
Process, starting from the knowledge of the state of the Holonic Controlled Process and of 
the group of activities to complete due to the method instantiated. The model of the 
mechanism of taking of decisions it is seen in a natural way as a model of hybrid system that 
is generally implanted as a Discrete Event Dynamical System (DEDS) described by the 
quintuple (X, U, Y, f (.), g (.)), where X is the group it didn't empty finite of states, U it is a 
group it didn't empty finite of the group or alphabet of controls, Y it is the group it didn't 
empty of output values. 
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Fig. 11. Holonic Supervised Process 
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3.3 Production unit holon 

In complex industrial processes, it is possible to have more than one supervisor where, 
conventionally, it is necessary a Coordinator for managing all of the supervisors. Figure 12 
presents this to facilitate the coordination tasks to carry out, all of the set of Supervised 
Process Holon, that we will call Holonic Supervised Production. 

Holonic Supervised Production 
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t 



Holonic Supervised Process 

(*,t/,r,f(.,.),g(.)) 



Tasks 



Fig. 12. Holonic Supervised Production 

Without losing generality, figure 13 presents the Holonic Supervised Production can have i 
models, each model has a combined m of tasks nominal instanced of the production method 
that can be reached with a combined j types of configurations which are adjusted under an 
approach determined by the supervisor with n of parameters. 



Tasks 
State 



t 



Holonic Supervised Production 
(X, U, Y, f(.,.), g(.)) 



Tasks 



Fig. 13. Holonic Supervised Production 

Thus, figure 14 shows the Production Unit Holon is conformed by a body that contains to 
the Holonic Supervised Production whose output is the state of the set of tasks agreed and 
the capacity of its resources (equipments, raw material, human resource, etc), the input are 
the set of planning tasks previously to complete the establish objective that it comes from an 
evaluated and negotiated recipe previously. 

The neck is conformed by the all tele-informatics architecture and applications that allow to 
detect events (they are able to capture, to try, to store, to adapt, event in events information) 
like to leave a recipe to project the set of tasks to the Holonic Supervised Production. The 
head or Fault Tolerance Coordinator, this conformed by the mechanism of taking of 
decisions capable of, starting from the knowledge of the state of the Holonicly Supervised 
Production and of the set the commitments with the clients and supply, to indicate that it 
will be manufactured and with which recipe previously was do it the negotiation. The 
model of the mechanism of decisions making is seen in a natural way as a model of hybrid 
system, that is generally implanted as a Discrete Event Dynamical System (DEDS) described 
by the quintuple one (X, OR, AND, f (.), g (.)), where X is the group it didn't empty finite of 
states, OR it is a group it didn't empty finite of the group or alphabet of controls, AND it is 
the group it didn't empty of exit values. 
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Fig. 14. Production Unit Holon 

4. Holonic enterprise 

A conventional enterprise is view like the set of production units associated to the chain of 
value of the productive process in a permanent manner since for this was constituted. A 
virtual enterprise is view like a set of enterprise associated to a chain of value of the 
productive process in a temporary manner since for this was constituted. Both use the chain 
of value of the productive process and for this reason the approach presented in this work, 
to each link of that chain of value we associate its a Production Unit Holon, see figure 15. 
Thus, a Holonic Enterprise this constituted for a body conformed by the set of Production 
Unit Holon associated to the chain of value of the productive process. 

Under our focus and in order to facilitate the management to carry out, all of the set of the 
Production Unit Holon that we will call to Holonic Coordinated Production is shown in 
figure 16. 
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Fig. 15. Value Chain vs. Production Unit Holon 
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In figure 17, the Holonic Coordinated Production can have i models, each model has a 
combined m of tasks nominal instanced of the production process that can be reached with a 
combined j types of configurations which are adjusted under an approach determined by 
the coordinator with n parameters. 
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Fig. 16. Holonic Coordinated Production 

The neck is constituted by the all of the tele-informatics architecture and applications, that 
allow to detect events (they are able to capture, to try, to store, to adapt, event in events 
information) like to leave a recipe to project the set of tasks to the Holonic Coordinated 
Production. 
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Figure 17. Holonic Coordinated Production 

As we can see in figure 18, the head or Fault Tolerance Manager, this conformed by the 
mechanism of taking of decisions capable of, starting from the knowledge of the state of the 
Holonic Coordinated Production and of the set the commitments with the clients and 
supply, to indicate that it will be manufactured and with which process previously 
negotiation. The model of the mechanism of decisions making it is seen in a natural way as a 
model of hybrid system that is generally implanted as a Discrete Event Dynamical System 
(DEDS) described by the quintuple one (X, OR, AND, f (.), g (.)), where X is the group it 
didn't empty finite of states, OR it is a group it didn't empty finite of the group or alphabet 
of controls, AND it is the group it didn't empty of exit values. 

4. Gratefulness 

• Scientific, Humanistic, and Technological Council (CDCHT), University of Los Andes, 
by mean of the project 1-867-05-02 -A, Production Unit Holon. 

• To the program France - Venezuela, ECOS NORD: Production Enterprise Nets. 



5. Conclusion and future work 

In a natural manner the business model, the chain of value, the product flow and the holonic 
approach allow us to capture a coherent model of the production process in all of their 
globallity and complexity giving origin to the shown embedded model. 
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The description invariant embedded presented of the Holon in production processes allows 
us to catch in a clear and precise way, the holonic recursive definition of the biology starting 
from the holarchy cell/ tissue/ organ like holonic functional basic unit of a body. 
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Fig. 18. Enterprise Holon 

This proposed recursive structure of the Holon to describe production processes allows us 

to present one scheme that is topologically the same, which should be configured in each of 

the instances required to model a company, including the established description of layers 

of the standard SP95. 

Our future work is devoted to complete a set of elaborated cases of study, where we show 

the implementation of these concepts in several domains. 

It is necessary to complete the description of the proposed Production Unit for each case. 

The negotiation mechanism must be implemented using a protocol that arrives to an 

agreement, which determines the cooperation among the units making part of the holonic 

enterprise. 
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1. Introduction 

In recent times, in-vehicle notifications have proliferated with a focus on the exhibition of 
technological prowess rather than the fulfilment of actual driving needs. In effect, 
information portrayed by automotive infotainment devices, while useful, is often ignored by 
the driver due to field of view limitations associated with traditional instrumentation 
panels. Not surprisingly, under poor visibility conditions and at motorway-level driving 
speeds, such systems do not effectively present useful information to the user. 
Contemporary Head-Up Display (HUD) experiments have focused on adapting the 
aviation-specific characteristics of HUDs to driver-specific needs, obsolescing functionality 
and simplifying operations where necessary. The more mature approach of these preceding 
works has revealed that although in-vehicle HUD technological advances have overcome 
most implementation issues, the related user-centred interface design is in its infancy 
prohibiting the HUD's unique features from being successfully exploited. Towards 
addressing this issue, in previous work, we have designed and implemented a functional 
prototype of a Human Machine Interface (HMI). Specifically, the proposed HMI system 
introduces a novel design for an automotive HUD interface, which aims to improve the 
driver's spatial awareness and response times under low visibility conditions. Particular 
emphasis has been placed on the prioritisation and effective presentation of information 
available through vehicular sensors, which would assist, without distracting, the driver in 
successfully navigating the vehicle. In order to evaluate the effectiveness of the proposed 
HUD system, we developed a driving simulator based on an open source racing program. 
Leveraging an open source solution has resulted in manageable levels of incurred expenses 
whilst allowing us to deliver a flexible simulation application for HUD evaluation. 
This chapter discusses the artificial intelligence (AI) as developed for the agent vehicles of 
our open source driving simulator. The simulator was explicitly designed to measure 
driver's performance when using the proposed HUD interface and compares its 
effectiveness to traditional instrumentation techniques. Intuitively, human cognition 
complexity poses the largest challenge for creating a model of life-like driver's behaviour for 
any type of traffic flow. Presuming that specific driving characteristics apply to all human 
drivers, as dictated by common sense, an attempt was made to form a generic reaction 
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pattern from a list of possible reactions to given situations. The AI embedded in 
participating vehicles in the simulations embraced certain assumptions, which are outlined 
later in the chapter. In this way the development time required for the AI routines was 
reduced by discounting rare occurrences that would indicate a severely misbehaving and 
irrational human driver. However, it was deemed necessary to develop AI controlled 
vehicles that could perform potential human miscalculations in order to enhance the realism 
of the simulation scenarios and their degree of mapping to real-life situations,. Imitation of 
human error by the robot vehicles, such as failure to brake on time, became a simulation 
feature that substantially enhanced the driver's immersion in the synthetic environment. 
Overall, this chapter elaborates on the modelling process of the agent vehicle's AI and 
discusses contemporary attempts to develop similar AI simulations for other simulation 
facilities. Notably, throughout this work, a concerted effort was made to ensure that the 
simulator scenarios were fair representations of potential real-world accident-prone 
situations; this has been made possible after attentive examination of the statistical data on 
driving scenarios provided by the Strathclyde Police Department situated in the City of 
Glasgow, UK. 

2. Proposed HUD simulation 

Early attempts to employ Head-Up Displays (HUDs) in automotive environments were 
greatly influenced by research in aviation and subsequently exhibited limitations and side- 
effects derived from the misuse of Human Machine Interaction (HMI) principles. Overall, 
the largely uncritical adoption of aviation HUD technologies held back the potential of 
automotive HUD use as conveyors of information provided by vehicular sensors. Even 
under such unfavourable conditions, however, research results have indicated an 
improvement in the driver's response time in some studies, which in turn hint on the 
potential of such systems. Recently, contemporary HUD experiments have focused on 
adapting the aviation specific characteristics of HUDs to driver-specific needs, obsoleting 
functionality and simplifying operations where necessary. The more mature approach of 
these preceding works has revealed that although in-vehicle HUD technological advances 
have overcome most implementation issues, the related user-centred interface design is still 
in its infancy prohibiting the HUD interface's capabilities from being successfully exploited. 
This study introduces a novel design for an automotive HUD interface which aims to 
improve the driver's spatial awareness and response times under low visibility conditions, 
offering only the crucial incoming information in a graphical representation manner. As 
such, a working prototype of a HMI has been designed and implemented to fulfil these 
requirements (Charissis et al. 2008a). Particular emphasis was placed on the prioritisation 
and effective presentation of information available through vehicular sensors, which would 
assist, without distracting, the driver in successfully navigating the vehicle under low 
visibility conditions (Charissis et al. 2008c). The harmonic collaboration between the human 
(driver) and machine (vehicle) elements has been supported by utilizing the machine as a 
collecting and distilling hub of information. Essentially, the human agent has been urged 
towards improved decision making through careful consideration of user characteristics and 
needs. That positive effect on the driver has been achieved by conveying the distilled 
information through the visual cues of the HUD interface reducing driver's response times 



Artificial Intelligence Rationale for Autonomous Vehicle Agents Behaviour 
in Driving Simulation Environment 



317 



in critical situations (Steinfeld & Green, 1995; Charissis et al. 2008b). In order to evaluate the 
system's effectiveness we used two simulation scenarios in subsequent user trials. These 
simulations required that substantial attention be given to the AI development in the 
computer-controlled vehicles. The following sections elaborate on the development of the 
driving simulator and the programming of the AI vehicles. 



3. Driving simulator as an evaluation tool 

Driving simulators are an indispensable tool used both in the automotive industry and 
academic research. Current state of the art simulators include the U.S. National Advanced 
Driving Simulator (NADS) (Papelis, 1994; Chen et al, 2001), the Swedish National Road and 
Transport Research Institute (VTI) (Nordmark et al., 2004; VTI, 2007), the Transport 
Research Laboratories (TRL, 2007) and the Leeds Advanced Driving Simulator (LADS, 2007) 
in the U.K. These sophisticated simulators can take into consideration numerous driving 
factors such as deceleration and acceleration forces, weather conditions etc., as well as be 
able to record, analyse and evaluate the results in real time (Kanto witch, 1999). However, 
the construction, upgrade and servicing costs of such elaborate hardware and software 
components can often be prohibitively high. Overall, and in view of conducting academic 
research, the task of designing and implementing a driving simulator, even one of low 
fidelity, can involve substantial financial overhead. Hence, academic institutes often rent the 
facilities of traffic research centres or automotive industry studios which offer specialised 
driving simulators for testing various automotive systems and devices. Intuitively, the 
existence of certain financial constraints may require the generation of alternative solutions. 




Figure 1. A screenshot of the (a) open source driving simulator and (b) the VR driving 
simulator developed based on the open source set-up. 

As the use of a driving simulator was necessary for the evaluation of the prototype HUD 
interface, we opted for an alternative low-cost approach. After careful consideration of the 
available funds and time constraints, the decision was reached to purchase off-the-shelf 
hardware components and develop the code on the open source TORCS platform (Wymann, 
2006; Centelles, 2006). This solution accounts for a manageable level of expenses whilst 
delivering a flexible simulation application for HUD evaluation purposes. 
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The reconstruction of accident events in a synthetic driving environment required the 
deployment of life-like driving virtual participants in the scene, thus a fleet of "intelligent" 
vehicles which could make maneuvering decisions in real-time and mingle orderly to form 
mixed traffic environments was deployed in the simulations examined here. The following 
section presents the challenges encountered during the implementation of realistic AI for the 
traffic vehicles involved in the events. 

4. Open Source Driving Simulator (OSDS) 

Recent advancements in processing capacity and graphic representation technologies for 
personal computers have significantly reduced the cost of realistic computer-generated 
environments. This vastly increased computational ability allows for vehicle and 
environment simulation that can reasonably approximate the definition performance of 
significantly more expensive driving simulator models (Hogue et. al. 2005). Singular PC 
units are able to support various features of a simulation process. Nevertheless, 
multitasking computation can potentially overload the system and affect the simulator's 
performance (screen refresh rate); therefore the simulation process has to be optimised with 
the view to meet the needs of the experiment (Rosenthal et al., 2003). 

A solution to the issue of singular unit performance can be achieved by clustering numerous 
PC simulators to effectively form a much more capable whole. In this case, PC simulators 
can divide the tasks and execute them separately; this combinatory approach amplifies the 
system's capabilities and may even reach the processing power of high-end servers as used 
for complex simulations. This newly formed breed of driving simulators has been adopted 
by high-end industrial and academic simulation facilities worldwide such as the Advanced 
Institute of Science and Technology (AIST) driving simulator at Tsukuba, Japan, the 
Transport Research Laboratory (TRL) in the UK and so on. However, even though the 
computational costs have been minimised, these facilities still uphold immense operating 
costs for upgrading, maintaining and running their systems. 

A more cost effective approach is the development on open source software able to simulate 
various vehicles and driving conditions. These specialised simulators can be easily 
customised to create a sufficiently realistic environment for testing various driving 
scenarios. Nonetheless, as the majority of such programs have been developed for gaming 
purposes, namely racing, they require substantial changes to the core of the programme in 
order to comply with a real driving scenario as discussed later in this chapter. The 
combination of off-the-shelf hardware (PC) and open source driving simulation software 
was, for our purposes, the most cost-efficient and flexible solution. The idea of creating such 
a custom-driving simulator benefited from input by various researchers from British, 
European and American universities (Charissis et al. 2006) 1 . A screenshot of the developed 
simulator in action is shown in Figure 2. 



1 This work has benefited from the feedback and contributions of a number of colleagues 
from academic institutions including the computing science departments of the University 
of Glasgow, the University of California, Berkeley, Glasgow Caledonian University and 
Aristotle University, Thessalonica. Where applicable each contribution is clearly marked 
and identified in this chapter. 
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Figure 2. Screenshot of the driving simulator with the HUD interface 



4.1 Open source driving simulator software development 

We conducted extensive market research which indicated that financial costs for software 
and hardware development would be considerable for building a medium fidelity driving 
simulator. Renting a simulator facility was also not a viable option as the high daily rates 
could not be covered by the research funds. It was therefore vital for the study to develop a 
custom simulating system with minimum expenditure. This necessity lead to collaborative 
work along with other researchers on the development of such a simulator. 
The open source 'The Open Racing Car Simulator (TORCS)" provided a suitable solution 
for the development of a custom driving simulator. Note that TORCS is a multiplatform, 3D 
car racing simulator based on OpenGL. This software formed the basis for the development 
of the OSDS. The source code was originally developed on the Linux platform but soon 
efforts were concentrated on the Windows operating system in order to ensure maximum 
compatibility with peripherals. The simulator's vehicles, i.e. the "robot" drivers, can be 
treated as agents with customisable AI. The purpose of programming the AI for each 
"robot" enables the participant to programme the attribute of their own racing vehicles 
(agents). As a result racing competitions can be held amongst pre-programmed vehicles by 
different authors on the Internet, or locally with the user racing against the computer 
vehicles. The navigation of the vehicles can be achieved through the use of a keyboard, 
mouse, joystick or steering wheel. Due to the racing nature of the software, the robot 
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vehicles imported into the OSDS required substantial reprogramming in order to adhere to 
the British Highway Code. The following section will elaborate on the simulation of the 
HUD interface and the environment visualisation. 

A. 2D HUD interface visualisation 

The functionality of the HUD interface components have been transferred to the simulated 
environment they it would purportedly be in a real vehicle. The symbols were programmed 
to follow the movement of the vehicles and alter their size and colour according to the 
relevant distance from the user's vehicle. The two dimensional projection of the OSDS did 
not offer depth of field, which could have further enhanced the realism of the experiment. 
However, the advanced AI of the robot vehicles and the accurate superposition of the HUD 
symbols enhanced the sense of presence; the derived results were based on genuine driving 
efforts to respond to the accident events. 

B. Simulation of low visibility conditions 

Given that the primary aim of the study was to measure and compare drivers' response 
times in adverse weather conditions, such environmental conditions had to be replicated 
accordingly. A survey of weather conditions literature defines that dense fog results in low 
visibility conditions, which produce significant traffic flow disruptions typically below the 
100m viewing distance mark (Cavallo et al. 1999; BBC Accidents, 2007). As such heavy fog, 
with visibility less than 50m, was simulated in four of the driving scenarios. Note that the 
visibility percentage could be adjusted via a simulation parameter achieve a satisfactory fog 
quality. Specific parts of the code were improved in order to simulate the desired fog 
conditions and realistic depth of field. The area of fog visualisation was revisited in our 
virtual reality driving simulator (VRDS) which particularly investigated the HUD's focusing 
distance as discussed in (Charissis et al., 2007b; Charissis & Naef, 2007). 

C. 3D visualisation 

To achieve a semblance to reality the testing track was designed and modelled based on a 
section of the M8 motorway in the outskirts of Glasgow, Scotland as depicted in Figure 3 
(item b). The simulation model was set to deal only with motorways with three lanes in each 
direction and one hard-shoulder lane. In order to reduce the amount of 3D geometry, the 
track was simplified to only one direction, in this case the route from Glasgow to Edinburgh 
(anticlockwise). A closed circuit was created by seamlessly connecting three main 
motorways thus forming a triangular track shape. This model did not incorporate ramps on 
motorways or intersections on rural roads. 3D visualisations of the important intersections 
and most recognisable landmarks were incorporated in the final model as illustrated in 
Figure 4. Such visual cues helped the drivers to recognise the environment. 
To populate the track, the 20 most popular vehicles, as seen on British roads, were modelled 
in low polygons geometry (gaming quality). Whereas the original program offered a wide 
choice of vehicles, the majority of them were racing cars and hence not appropriate for this 
experiment. The 3D objects and track were modelled using Alias Maya 6.5, and after a 
distilling process of exports in various formats, they were imported into the simulator using 
Track Editor and AC3D software. Additional alterations to the XML code were needed to 
allow compatibility with Windows XP. 
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Figure 3. (a) Introductory screen of the OSDS, (b) Simulation track based on real motorway 
routes. 







Figure 4. (a) Landmarks in real environment and (b) in the simulated track 



4.2 Open source driving simulator hardware 

The development of a cost efficient simulator could be mainly achieved by maintaining the 
costs of the hardware at minimum levels. However the keyboard or mouse solution did not 
seem appealing, as it could subconsciously be reminiscent to the user of a computer game. 
As a full-windshield HUD evaluation required a " close-to-rear driving environment it was 
deemed essential to purchase an " off- the shelf' steering wheel (Logitech Driving Force Pro 
GT4). This hardware item is categorised in the range of gaming products, however that 
particular steering wheel model offered a quite realistic feel (three rotation circles from left 
to right) and a well-implemented force feedback function (realistically transferring road 
bumps and the feeling of vehicle's drifting or braking). Note that his component required 
additional programming as the open source programme failed at first to recognise the 
device automatically. The steering wheel was also accompanied with a bundle of other 
equipment which included foot pedals for accelerator and brake (see Figure 6). 
The software was run on a custom built PC with dual 64bit processors (AMD Opteron-242) 
and a Quadro FX1100 graphics card. A frame-rate of 60-90 fps was effortlessly maintained 
by the system. Although the simulator could perform well on a low specification laptop, this 
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powerful dual processor system was built with the view to accommodate simultaneously 
more than one driver for future experiments. 

Emphasis was also placed on the implementation of a driving cockpit environment; a real 
driving seat (a Gamepod, "bucket seat") formed in large part the driver's virtual "cockpit". 
The seat stood on a railing system on the top of a metallic structure which also supported 
the minimalistic dashboard and the steering wheel. The railing system allowed users to alter 
the positioning of the seat, the steering wheel and the pedals to match their physical 
dimensions and driving preferences. 

The relevant hardware and software equipment had to be accommodated in a facility that 
could host two-week long simulation trials and experimentation as driver reactions were 
recorded. The Glasgow Caledonian University generously offered the E-motion Lab of the 
Mathematical and Computing Sciences Department which was explicitly equipped with 
numerous recording and observation devices such as video cameras, eye-fixation recorders, 
motion detectors and a fully developed observation suite and control room. The academic 
staff aided us greatly in setting up the connection of the driving simulator devices to the 
facility's additional equipment. 

The observation suite hosted the driving simulator, which was projected on the lab's 
Panasonic 42 inch plasma screen positioned approximately 2m away from the driver's seat, 
as shown in Figure 5. The control room was equipped with monitors showing, in real time, 
the video stream from the cameras in the simulation area. The focus and the position of the 
cameras could be adjusted also in real time through a joystick on the control console in the 
observation room. The two videos could be seen either on separate monitors or both on the 
same window (screen on screen) thus enabling the researcher to observe the driver's driving 
process (via a screen camera) and his/her facial expression and posture (via a camera 
focused on the user) The following section presents the challenges encountered during the 
implementation of realistic AI for the traffic vehicles involved in the events. 

5. Accident scenarios simulation 

As Boer et al. (Boer & Ward, 2003) define it, an event is a situation requiring a corrective 
response such as, say, a situation arising when safety margins are violated. In order to 
evaluate the driver's response to accident events, the AI vehicles employed in the simulation 
scenarios had to generate the right circumstances, which would eventually lead to such an 
event occurring. 

Note that the agents involved in the simulated scenarios were programmed to react and 
manoeuvre according to the flow of traffic and external events, whilst demonstrating 
natural driving behaviour. Every vehicle would obey certain rules which "urged" it to 
imitate its real life counterpart while its overall behaviour was influenced by several factors. 
In particular, each vehicle calculated, in real time, information drawn from its surroundings 
(i.e. other traffic vehicles) as well as information about visible accident hotspots such as 
enclosing side barriers, bridges, traffic cones and so on. The above information was 
clustered in the agents' intelligence creating logical links among possible events and 
responses, thus forming the framework for each individual agent's AI. 

The following subsection elaborates on the driving scenario development. Section 5.2 
explicitly analyses the macroscopic and microscopic driving scenario approaches, which 
consequently form the generic AI attributes of the vehicle agents presented in Section 5.3. In 
turn Section 5.4 presents the drivers' patterns infused in the agents' driving attributes. 
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Figure 5. (a) Open Source Driving Simulator (OSDS) schematic top view, (b) example of 3D 
visualisation of simulated vehicle, (c) driving seat and positioning overview 



5.1 Driving scenario development 

The development of traffic scenarios was accomplished through careful inspection of data 
provided by actual traffic police reports. These statistics and planning diagrams aided in 
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predicting drivers 7 possible reactions (SPD, 2004). Subsequent analysis showed that two 
particular car-following scenarios occur fairly frequently and reveal a high fatality rate. A 
detailed description of both such scenarios is given later in this subsection; however, before 
an outline of the two scenarios may be presented, it is useful to denote certain constraints 
and considerations evident in both. Firstly, note that for validation purposes, the movement, 
speed and distances between the vehicles had to adhere to the British Highway Code. 
Moreover, in order to enhance the realism of the simulation scenarios and their degree of 
mapping to real-life situations, the AI controlled vehicles had to perform potential human 
misjudgements. Imitation of human errors by the robot vehicles (i.e. failure to brake on 
time) is a simulation feature that substantially enhances a driver's immersion in the 
synthetic environment (Allen et. al., 1997). As the authors argue in Park et. al. (2004), the 
driver has to be challenged in order to react and produce driving skills that would normally 
apply in a real accident situation. Overall, two common driving situations of a "car- 
following" scenario were developed for the test-bed experiments based on observations and 
accident prompt strategies produced in previous research (Daganzo, 1999; Smith et al., 
2003). All the scenarios were presented in a motorway environment with heavy fog 
featuring low visibility (clear view being available at under 50m distance). 
The first scenario used in this work is a variation of a generic car-following model described 
in previous work (Brackstone & McDonald, 2003). In that work, the user drives along the 
motorway and after having travelled a distance of 2km, the lead vehicles brake abruptly, 
causing following vehicles to decelerate rapidly. Intuitively, this event increases 
substantially the chances of vehicle collision. A previous study focusing on mapping of 
driver's possible reactions in similar car following accident scenarios (Smith et. al., 2003) has 
suggested that a driver's performance is comprised of four driving states: low risk, conflict, 
near crash and crash imminent, corresponding to four different warning levels respectively. 
As such, the first scenario was developed along these guidelines in order to evaluate the 
HUD's ability to effectively convey these four collision states to the driver. Analysing the 
driver's performance under such discrete time segments, each mapping onto these four pre- 
collision periods, has provided the study with the ability to identify, for every stage, the 
impact of the added HUD information over a typical HDD. 

The second scenario is a variation of the original car-following situation in which the user 
drives for 5km following a lead vehicles' group, without any major occurrences taking 
place. After 5km the road forms a sharp turn (120 degrees) underneath a bridge thus 
creating an accident-prone situation. The difficulty of the scenario is amplified by the 
adverse weather conditions, in this case heavy fog, which dramatically decrease the driver's 
visibility, and by the addition of slow moving traffic congestion positioned at the exit of the 
turn. As before, the four states of collision warnings observed in other work (Smith et. al., 
2003), were infused in the rationale of the potential accident driving scenarios. In addition, 
collision-prone issues such as sharp turn negotiation and traffic congestion were also 
employed in order to investigate driver's performance with and without the assistance of 
the proposed HUD interface. 

In both scenarios the user is forced to respond instantly, either by manoeuvring around the 
accident point or by braking. The robot vehicles involved in the scenarios are programmed 
to minimise the possibility of accident avoidance, as the experimental focus was to gather 
detailed measurements of drivers' response times and distance from the lead vehicle with 
respect to the accident event. A schematic overview of both scenarios is shown in Figure 6. 
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Figure 6. Simulation Scenarios 

5.2 Macroscopic simulation approach 

Investigation of driver behaviour modelling research demonstrated that the majority of 
high-fidelity traffic simulators utilise the macroscopic simulation method (Kuhl et aL, 1995; 
Cremer, 1995; Papelis & Bahauddin, 1995). Predominantly, this method exploits 
mathematical models - often originated from fluid dynamics - that treat vehicles as particles 
in a homogenous flow. A sophisticated version of the macroscopic simulation method can 
identify three or more groups of vehicles inside the traffic flow which share identical 
characteristics (typically size and speed) such as cars, motorbikes and buses. This allows for 
some degree of control over the simulated traffic as these three main groups are 
characterised by different driving speeds and movements (Ehlert & Rothkrantz, 2001). Yet, 
these groups do not have any differentiating attributes between individual vehicles in the 
same group; individual driving characteristics are ignored in favour of simulation of very 
large numbers of vehicles moving in large urban or motorway complexes. Therefore the 
utility of the macroscopic approach lies mostly within generic traffic models and estimation 
of traffic pattern rather than the drivers' individual behaviour. 

Evidently, imitation of real life occurrences requires collection of different vehicles, 
representations of their driving patterns and creation of potentially unpredictable traffic 
flows. Yet, a number of driving constraints should be considered and a simplification model 
should be applied in order to ease the simulator's calculating demands. Typically, when 
considering a macroscopic simulation scenario, variations therein depend on the number of 
different groups of vehicles. A group is defined as a set of vehicles that have the same 
driving pattern within a driving session. The scenarios used in this work were initially 
developed using the macroscopic method to define traffic flow complexity. In this 
simulation we opted for clustering of the vehicles into two main groups, named "Stop- 
aheads" and // Jammies ,/ , better control over the generation of the accident scenarios was 
achieved (Charissis et aL, 2007a). These two groups had been particularly developed to play 
different roles in the accident simulation. Although both of them were seamlessly integrated 
into the generic flow, at a specific moment of the simulation they were scheduled to act 
according to a master-plan that initiated a potentially accident-causing event. At a 
predetermined moment the group of lead vehicles (named "Stop-aheads") brakes abruptly 
challenging the user to respond immediately. The second group (named // Jammies ,/ ) mainly 
populated most of the motorway and in some cases developed condensed traffic sections or 
mini traffic jams. However, the integration of the two groups forming the traffic flow was 
not identifiable by the driver as an artificial construct because the robot vehicles were 
developed with additional singular driving characteristics, which provided the impression 
of autonomous behaviour. These are presented in Figure 7, which shows a diagram of the 
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Figure 7. A diagram of the main robot vehicle categories. 

5.3 Microscopic simulation approach 

In addition the microscopic method was utilised to embed the AI into the robot vehicles and 
to create different driving profiles (one for each agent). This technique is predominantly 
used for accident reconstruction where specific agents have to perform particular roles in 
order to effectively reproduce the circumstances that created the accident. This in-detail 
approach could not be achieved solely by the macroscopic approach, as it does not address 
the impact to individual vehicles' problems on the traffic (accidents, vehicle break downs 
etc). Contrary it solely investigates the impact of such individual incidents on the whole 
traffic flow (Ehlert & Rothkrantz, 2001) rather the individual behaviour of neighbouring 
vehicles. Thus the individual behaviour of each agent catered for realistic interactions 
between the robots regardless of their group identity. 

In the simulator the agent's behaviour can be altered by tweaking a set of parameters such 
as general speed, top speed for session, speed close to turns or linearity to lane (distance 
from centre of lane). Nevertheless, certain restrictions had to be pre-programmed for all the 
agents in order to keep autonomous behaviour within acceptable and law-abiding levels. 
Summarising, both macroscopic and microscopic traffic simulation systems were embedded 
in the driving simulator. This combinatory simulation approach has been favourably rated 
in the literature for its successful realistic depiction of real traffic interactions (Shiraishi et al., 
2004; Sahraoui & Jayakrishnan, 2005). As a measure of comparison it should be stated that 
the simulator used in this work handled approximately twenty vehicles; state of the art 
simulators which facilitate virtual traffic using real time generated scenarios may 
incorporate up to 100 participating vehicles created and controlled independently (Suda, 
2006). However, a repetitive flow of vehicles inside the looped triangular circuit used in 
both scenario setups provided the illusion of constant moderate traffic of an endless number 
of travelling vehicles. 

6. Embedding Al into vehicle agents 

Intuitively, anticipation of human reactions presents a grandiose challenge for creating a 
model of life-like driver's behaviour for any type of traffic flow. Hypothesising that specific 
driving characteristics apply to all human drivers, as dictated by "common sense", an 
attempt was made to form a generic reaction pattern from a list of possible reactions to 
given situations (Yamanaka et. al., 2005; Sukthankar et al., 1996). The AI embedded in 
participating vehicles in the simulations embraced certain assumptions, which are outlined 
below. In this way the development time required for the AI routines was reduced by 
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discounting rare occurrences that would indicate a severely misbehaving and irrational 
human driver. 

6.1 Simulated driving constraints 

Exhaustive investigation into drivers' possible reactions identified a number of constraints 
and rules that define the unpredictable factors involved in such simulation model which 
seeks to re-enact specific accident, real-life scenarios. The constrains identified could be 
divided in two main categories namely; soft and hard constrains, which are introduced in 
turn. 

The soft-constraints in a driving scenario are set either by road rules (the motorway code), 
which are obeyed in normal situations (i.e. avoiding collision with the neighbouring 
vehicles) or by the agent's behaviour (i.e. speed limited by 'fear of crashing'). Minor 
deviations in soft constrains are favourable in a simulated environment in order to depict 
closely the real-life behaviour of various drivers. However it is worthwhile to note that such 
soft-constraints are exactly the ones violated heavily in crash situations. 
On the other hand, there are also certain hard-constraints (for instance driving opposite to the 
direction of the traffic flow), which if violated would render a model void of realism. The 
simulation parameters can therefore vary and a semblance to reality may be maintained as 
long as hard-constraints are not violated. If a particular set of simulation parameters stays 
within the soft-constraints then normal driving conditions are denoted; alternatively, any 
violation of these signifies a particular accident-prone traffic set-up such as the one in the 
car-following scenario where a collision might occur due to adverse weather conditions. 

6.2 Vehicles' agents categories 

Both driving accident events have been simulated by the agent vehicles that populated the 
track. In order to attain the seamless integration of the robot vehicles into the traffic flow, a 
large part of the initial open source code had to be re-written as it was originally designed 
for racing purposes only. The driving behaviour of 20 vehicles was individually 
reprogrammed to follow the Highway Code (e.g. speed limits, lane keeping, signalling, etc.). 
In the first scenario (sudden braking of the lead vehicles), the robot vehicles were divided 
into 3 groups called "traffic waves". The primary purpose of these waves was to 
intentionally constrain the users from exceeding the speed limit, which eventually 
implicated them in the car following accident events. The secondary objective of the waves 
was to augment the sense of realism. In particular, when the simulation started, the test car 
was positioned amongst the third wave of vehicles ensuring that the driver would be 
accompanied at all times during the simulation. At a predetermined point, the lead vehicles 
of the second wave were programmed to brake abruptly, thus instigating the accident 
scenario. The succeeding vehicles would then respond randomly to that event either by 
braking on time or by colliding with the front vehicles. Given that limited visibility due to 
simulated thick fog, the driver would have little time to decide on what action to perform. 
There were two common reactions: harsh braking or manoeuvring around the stopped 
vehicles via the hard shoulder lane. In the case where collision with the second wave was 
avoided, the first wave of vehicles repeated the same scenario after 300m therefore 
maximising in this way the possibilities of accident involvement. 

Similarly, for the purposes of the second scenario, two traffic wave groups were formed: the 
accompanying group and the congestion group. Whilst the accompanying vehicles had the 
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same driving behaviour as in the first scenario, the congestion group had been allocated 
virtually motionless behind a sharp left turn, under a complex of bridges. This accident 
scenario can be equally hard to avoid when visibility is low as it is difficult to identify the 
sharp turn and traffic congestion in advance. Achieving a high level of realism for the 
scenarios was the most important element for the evaluation of the proposed HUD. The 
sense of immersion can be initially achieved by the presence of a sufficient number of 
vehicles on the track with a distinguishable variation of behaviours comparable to the 
variation found in reality. 

A particularly interesting re-inaction of that driving pattern variation was achieved in the 
second scenario. As the user is driving along in the light traffic she encounters a sharp bend 
under a bridge and traffic congestion at the exit of the curve. It is worth noting that this 
scenario is actually a depiction of a real problem, so there were no other pre-programmed 
accident events taking place (like abrupt braking of the lead vehicles in the first scenario). 
The robot vehicles forming the traffic around the driver were expected to improvise and 
react in an appropriate manner. A variation of AI reactions was recorded when robot 
vehicles attempted to stop behind the static traffic: some tried to manoeuvre around and 
utilise the hard-shoulder lane while others crashed into the rear of the static traffic. The 
interaction amongst the different agents and groups considerably enhanced the feeling of 
realism as the driver could witness a realistic conclusion to the event. 

As described in the microscopic method of the simulation, different categories of agents 
were deployed depending on their role in the forthcoming events. The major agents strongly 
instigated the particular key event (e.g. an imminent collision) and the observer agents 
indirectly influenced the final event. The ambiguity in any model is that in real life every 
driver can alter her category status with regards to a hypothetical accident. However, on 
hindsight of a particular key event the majority of the drivers behave in a similar fashion so 
as to be grouped appropriately. 

6.3 Driving pattern investigation 

In order for the simulation to become more realistic, it is often useful to add a degree of 
randomness to the distances between cars at the initial simulation setup as well as provide 
some variation to the simulation time. 

Evidently the simulated groups have three specified soft-constraints, nevertheless, there are 
some hidden constraints generated from the vehicle models and road model. As more 
groups of vehicles with specific purpose are introduces, the number of constraints on 
driving behaviour increases and some control is introduced in the environment, leading to a 
'test scenario' from which conclusive judgements can be drawn. When unpredictable events 
occur in a scenario it becomes harder to deduce the reason for the key event happening - as 
the user might get distracted by 'non-key events' taking place due to vehicles' variation; 
these affect the driver's reaction to the key event. 

However, such variations do exist in reality due to the numerous differences in driving 
behaviours of real drivers. We discovered that to achieve both realistic variation and 
sufficient control necessary for the key event to take place, the driving time has to be 
sufficiently long. In this way, each group in a simulation can exhibit "personality" whilst 
their influence gets slightly diluted over time, so that their behaviour will not appear more 
peculiar than the key-event. 

While developing the simulator and its associated AI we also took into consideration the 
possible design of the networking and systems backend which would be necessary for the 
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implementation of our HUD design in actual vehicles. We have argued, in related work, that 
the detection of the headway and positioning of the neighbouring vehicles can be 
accomplished by a collaborative Mobile Ad-hoc Networking System (MANETS) between 
vehicles and with the aid of GPS and sensors (Charissis & Papanastasiou, 2006). 

7. Al simulation challenges and solutions 

During the development of the driving simulation environment, the agent vehicles 7 
autonomous reactions had been extensively tested in order to identify potential issues. Initial 
presumptions with regard to different driving patterns resolved most anticipated issues, yet 
some participants 7 unexpected behaviour reactions required more creative solutions (Green, 
2005). In order to address the problems that occurred during the preliminary trials, we 
employed fundamental techniques used in similar cases in AI such as the Cognitive 
Simulation introduced by Newell, Shaw and Simon at the RAND Corporation and at Carnegie 
Institute of Technology (Dreyfuss, 1992). Originally this approach involved the collection of 
protocols from human subjects which were subsequently analysed to discover the heuristics 
that these subjects employed (Dreyfuss, 1992). More specifically, the following two solutions 
were applied to retain the driver amongst at least one of the traffic waves. 

7.1 "Bandit-Driver" phenomenon 

This phenomenon typically occurs in a driving simulation environment when the user reacts 
as if driving a " racing computer game". In order to avoid such behaviour which alters 
considerably the real results, the users had been instructed to follow the Motorway Code for 
the entire duration of the experiment. Yet a number of users exceeded the speed limit 
substantially. This, however, resulted in an event-free journey as they had over-taken all the 
traffic waves. After experimenting with different speeds and chase techniques for the 
driving agents, we decided that one of the vehicles should play the role of the "super-car". 
Therefore one of the vehicles had been given the ability to chase and dangerously over-take 
the "bandit-driver" forcing him/her to slow down until the rest of the traffic would merge 
into the picture. That ability was triggered by a chosen agent as soon as the "bandit-driver" 
was out of range of any possible event (1km). 

7.2 "Timid-Driver" phenomenon 

The opposite of the previous case was the exceptionally slow driver, also referred to as a 
"timid-driver" (Daganzo 1999). Even when the robot vehicles were decelerating 
significantly, such drivers were travelling with a speed of less than 30 km/h thus 
maintaining a considerable distance from the preceding main traffic. This resulted once 
more in the participant missing out the accident events. The remedy to this was the 
incorporation of the "fear factor". A model of a black truck participated in that role. The 
truck was travelling at a conveniently slow speed, following the slow vehicle in close 
proximity (not visible due to the fog-effect). So, when the user was permitting a greater 
distance than 1km from the last leading vehicle, the agent of that robot vehicle had been 
programmed to initiate a close-vicinity pursuit. If the driver persisted with the same driving 
pattern the truck would alarmingly sound its horn. 

We considered that it might seem reasonable to exclude the results of both "bandit" and 
"timid" users from the final analysis. However, driving behaviours similar to the ones 
described above exist in real life; these extra agents were developed in order to embrace a 
wider breadth of driving patterns. 
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7.3 "Follower" phenomenon 

This category of drivers typically followed closely the movements of the lead agent vehicles. 
As such they increased their level of anticipation significantly by simply imitating the 
driving behaviour of the simulated vehicles. Although such behaviour appears in real-life as 
well and in some respect may even be desirable, during simulation it does not accurately 
reflect the driver's real intentions. This occurs as drivers feel that the simulation experiment 
will evaluate their own driving abilities instead of the effectiveness of the interface. In order 
to avoid such issues we enhanced the ability of the "jammies" to overtake and partially 
merge with the lead vehicles. That position exchanging and multiple unexpected 
manoeuvres blur significantly the roles between the agent vehicles for the participant. 

8. Conclusions 

This chapter presented the rationale and development process of an open source driving 
simulator which was exclusively created for the evaluation process of a prototype HUD 
interface. Financial limitations led to the exploration of new avenues for evaluating the 
prototype HUD. The simulator hardware was built using off-the-shelf components and the 
software was developed through customisation of an open source code game engine. 
For evaluation purposes, equal emphasis was placed on was the development of an 
appropriate testing environment and suitable driving scenarios. To achieve a realistic mix of 
agent behaviour, the AI infused in the vehicles 7 agents was thoroughly customised and 
designed to facilitate the accident events created to gauge driver reactions. 
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1. Introduction 



There have been many researches about humanoid robot motion control, for example, 
walking pattern generation (Huang et al, 2001) (Kajita et al, 2002, 2003), walking control 
(Choi et al, 2006) (Grizzle et al, 2003) (Hirai et al, 1998) (Kajita et al, 2001) (Kim & Oh, 2004), 
(Lohmeier et al, 2004) (Park, 2001) (Takanishi et al, 1990) (Westervelt et al, 2003), running 
control (Nagasaki et al, 2004, 2005), balancing control (Kajita et al, 2001) and whole body 
coordination (Choi et al, 2007) (Kajita et al, 2003) (Sentis & Khatib, 2005) (Goswami & 
Kallem, 2004) (Harada et al, 2003) (Sugihara & Nakamura, 2002). Especially, the whole body 
coordination algorithm with good performance becomes a core part in the development of 
humanoid robot because it is able to offer the enhanced stability and flexibility to the 
humanoid motion planning. In this chapter, we explain the kinematic resolution method of 
CoM Jacobian with embedded motion which was suggested in (Choi et al, 2007), actually, 
which offers the ability of balancing to humanoid robot. For example, if humanoid robot 
stretches two arms forward, then the position of CoM (center of mass) of humanoid robot 
moves forward and its ZMP(zero moment point) swings back and forth. In this case, the 
proposed kinematic resolution method of CoM Jacobian with embedded motion offers the 
joint configurations of supporting limb(s) calculated automatically to maintain the position 
of CoM fixed at one point. 

Also, a design of balancing controller with good performance becomes another important 
part in development of humanoid robot. In balancing control, the ZMP control is the most 
important factor in implementing stable bipedal robot motions. If the ZMP is located in the 
region of supporting sole, then the robot will not fall down during motions. In order to 
compensate the error between the desired and actual ZMP, various ZMP control methods 
have been suggested; for example, direct/ indirect ZMP control methods (Choi et al, 2004) 
(Kajita et al, 2003) and the impedance control (Park, 2001). Despite many references to 
bipedal balancing control methods, research on the stability of bipedal balancing controllers 
is still lacking. The exponential stability of periodic walking motion was partially proved for 
a planar bipedal robot in (Grizzle et al, 2003) (Westervelt et al, 2003). Also, the ISS 
(disturbance input-to-state stability) of the indirect ZMP controller was proved for the 
simplified bipedal robot model in (Choi et al, 2004, 2007). In this chapter, we will explain the 
balancing control method and its ISS proof which were suggested in (Choi et al, 2007). The 
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suggested method offers the stable balancing control as well as the function of whole body 
coordination to humanoid robot. Also, the suggested control method includes both the 
CoM/ZMP trajectory tracking control and CoM/ZMP set-point regulation control based on 
the kinematic resolution of CoM Jacobian with embedded task motions. Due to the 
modeling uncertainties and the complexity of the full dynamics of a bipedal walking robot, 
we will represent the dynamic walking robot as a simple rolling sphere model on a 
constraint surface. And then the ISS is proved for the simplified model of bipedal robot. 
This chapter is organized as follows: section 2 introduces a simplified model for bipedal 
walking robot, section 3 explains the kinematic resolution method of CoM Jacobian with an 
embedded task motion, section 4 suggests the balancing controller for humanoid robot and 
proves its ISS for the simplified bipedal robot model, section 5 shows the experimental 
results about the stable balancing functions obtained by using the kinematic resolution 
method of CoM Jacobian with embedded motion, and section 6 concludes the chapter. 

2. Rolling sphere model 

The bipedal walking mechanism is an essential part of humanoid robot. Since humanoid 
legs have high degrees of freedom for human-like motion, it is difficult to use their full 
dynamics to design controller and to analyze stability. Therefore, we will simplify the 
walking related dynamics of bipedal robot as the equation of motion of a point mass 
concentrated on the position of CoM as shown in Fig. 1. 



Body Center 
Frame 




World Coorinate 
Frame 



Fig. 1. Humanoid Robot 

First, let us assume that the motion of CoM is constrained on the surface, z = c z , then the 
rolling sphere model with the concentrated point mass m can be obtained as the simplified 
model for bipedal robot as shown in Fig. 2. In Fig. 2, the motion of the rolling sphere on a 
massless plate is described by the position of CoM, c= [c x9 c 9 c 2 ] T , and the ZMP is described 

by the position on the ground, p = [p x ,p y , Of . 
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X 



Fig. 2. Rolling Sphere Model for Humanoid Robot 

Second, the equations of motion of the rolling sphere (mass =m) in Fig. 2 are expressed on 
the plane z = c z by: 



r x = mgc y - mc y c z 
r y = -mgc x + mc x c z 



(i) 

(2) 



-mcc„ + mcc^ 



where g is the acceleration of gravity, c z is a height constant of constraint plane and r t is 
the moment about z-th coordinate axis, for i = x,y,z . Now, if we introduce the conventional 
definition of ZMP as following forms to Eq. (1) and (2): 



P x =- 



mg 



Py 



mg 



then ZMP equations can be obtained as two differential equations: 
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Px 



(3) 



Py 



(4) 
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c i 




"0 


r 


c i 
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" 


dt 


c. 




k 





c. 




W 



Actually, above equations can be easily obtained by finding the position of zero moment on 
the ground in Fig. 2 like these: 

YM P = -mg(p y -c y )~ (mc y )c z = 

y 

H M P X = m S(P X ~ C x) + ( mij X ) C z = - 

Also, the state space realization of ZMP equations (3) and (4) can be written as: 



Pt> 



for i = x,y , where co n = ^g/c z means the natural radian frequency of the simplified biped 

walking robot system. These state space equations describe the relation between the 
dynamics of CoM and the ZMP. Above equations will be used to prove the stability of the 
balancing controller in the following sections. 

3. Kinematic resolution method for balancing 

In this section, we will explain the kinematic resolution method of CoM Jacobian with 
embedded motion, ultimately for humanoid balancing. Let a robot has n limbs and the first 
limb be the base limb, for example, n=4 for humanoid robot except the neck. The base limb 
can be any limb but it should be on the ground to support the body. Each limb of a robot is 
hereafter considered as an independent limb. In general, z-th limb has the following relation: 

°x. =° J i q i 
for i = l,2,---,n, where °x i e 9t 6 is the velocity of the end point of z-th limb, q t e 9?" is the 

joint velocity of z-th limb, V. e 9? x "' is the usual Jacobian matrix of z-th limb, and n i means 
the number of active links of z-th limb, at least, n i > 6 for the limb related with leg. The 
leading superscript o implies that the elements are represented on the body center 
coordinate system shown in Fig. 1 and Fig. 2, which is fixed on a humanoid robot. 

3.1 Compatibility condition 

In our case, the body center is floating, and thus the end point motion of z-th limb about the 
world coordinate system is written as follows: 



x i =XT x x o +X o °J i q i 



(5) 
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where x o =[r o T ;a> o T ] T e$i 6 is the velocity of the body center represented on the world 
coordinate system, and 



X ; 






:9f 



^3 *3 

is a (6 x 6) matrix which relates the body center velocity and the z-th limb velocity. I 3 and 
3 are an (3 x 3) identity and zero matrix, respectively. R o °r . is the position vector from the 
body center to the end point of the z-th limb represented on the world coordinate frame. 
[(•)x] is a skew-symmetric matrix for the cross product. The transformation matrix X o is 



X n 



3 
R 



e<R 6 



where R o e lR 3x3 is the orientation of the body center represented on the world coordinate 

frame, and hereafter, we will use the relation J. = X o ° J. . Also, the concrete proof of Eq. (5) is 

in appendix 9.1. 

All the limbs in a robot should have the same body center velocity, in other words, from Eq. 

(5), we can see that all the limbs should satisfy the compatibility condition that the body 

center velocity is the same, and thus, z-th limb and ;-th limb should satisfy the following 

relation: 



X i (x i -J i q i ) = X j (x j -J j q j ). 



(6) 



From Eq. (6), the joint velocity of any limb can be represented by the joint velocity of the 
base limb and Cartesian motions of limbs. Actually, the base limb should be chosen to be the 
support leg in single support phase or one of both legs in double support phase. Let us 
express the base limb with the subscript 1, then the joint velocity of z-th limb is expressed as: 



q t = Jtx. -JlX a (x x - J x q x ), 
for i = 2,---,n , where Jt means the Moore-Penrose pseudoinverse of J. and 



(7) 



I, = x: l X, 



7 3 [* (V-V> 
0, /, 



Note that if a limb is a redundant system, any null space optimization scheme can be added 
in Eq. (7). With the compatibility condition of Eq. (6), the inverse kinematics of humanoid 
robot can be solved by using the information of base limb like Eq. (7), not by using the 
information of body center like Eq. (5). 



3.2 CoM Jacobian with fully specified embedded motions 

Now, let us rewrite the conventional CoM Jacobian suggested in (Sugihara & Nakamura, 
2002) as follows: 
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c = r n +a>, 



>oX(c-0 + Z R »°JJr 



(8) 



where n is the number of limbs, c is the position vector of CoM represented on the world 
coordinate system, namely, c = [c x ,c y ,c z ] T , °J C eSR*"' means CoM Jacobian matrix of z-th 

limb represented on the body center coordinate frame, and hereafter, we will use the 
relation J c = R °J C . Also, the concrete proof of Eq. (8) is in appendix 9.2. 




Fig. 3. Position of CoM of the k-th link in i-th limb 

Remark 1 The CoM Jacobian matrix of i-th limb represented on the body center frame is expressed by 



°A=I> 



d°c iti 
dq i 



(9) 



where °c ik e $R 3 means the position vector of center of mass of k-th link in i-th limb represented on 

the body center frame as shown in Fig. 3 and the mass influence coefficient of k-th link in i-th limb is 
defined as follow: 



Ma = - 



Z2>a 



(10) 



where m uk is the mass of k-th link in i-th limb. Also, the systematic derivation of CoM Jacobian 

matrix ofEq. (9) is in appendix 9.3. 

The motion of body center frame can be obtained by using Eq. (5) for the base limb as 

follows: 



XAx.-J.q, 
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h [*>,x] 
3 /, 



(11) 



where J v and J a are the linear and angular velocity part of the base limb Jacobian J x 

expressed on the world coordinate frame, respectively. Now, if Eq. (7) is applied to Eq. (8) 
for all limbs except the base limb with subscript 1, the CoM motion is rearranged as follows: 



c = t + co o x ( c - r o ) + J q x + %T c jr(x. - X iX x x ) + £j c j;X n J x q x . 



(12) 



Here, if Eq. (11) is applied to Eq. (12), then the CoM motion is only related with the motion 
of base limb: 



c= r x + co x xr cX - 



J Vx <7i + r cl x J q x + J q x + ^V c . J- (*,• " X iX x x ) + ^J c j;X n J x q x (13) 



where r cX =c-r x . Also, if the base limb has the face contact with the ground (the end-point 
of base limb represented on world coordinate frame is fixed, x x = , namely, r x = , co x = ), 
then Eq. (13) is simplified as follows: 

c ~ Y/ Ci Jt*i = ~ J v x 4i + r cx X J co x <li + J c x <l\ + Yj C[ J i X il J A • 

i=2 ' i=2 

Finally, 3 x ^ CoM Jacobian matrix with embedded motions can be rewritten like usual 
kinematic Jacobian of base limb: 



C fsem ^fsem #1 ' 



(14) 



where 






(15) 



isem v, cl <y, c, / ^ c. i i\ 1 

i=2 

Here, if the CoM Jacobian is augmented with the orientation Jacobian of body center 
( co o = - J m q x ) and all desired Cartesian motions are embedded in Eq. (15), then the desired 

joint configurations of base limb (support limb) are resolved as follows: 



%c 



fsem 

-J. 



°fsem,rf 



(16) 



where the subscript d means the desired motion and 
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-TJJtK, 



(17) 



All given desired limb motions, x. d are embedded in the relation of CoM Jacobian, thus the 

effect of the CoM movement generated by the given limb motion is compensated by the 
base limb. The CoM motion with fully specified embedded motions, c fsemd , consists of two 

relations: a given desired CoM motion(the first term) and the relative effect of other 
limbs (the second term). The CoM Jacobian with fully specified embedded motions, J fsem 
also consists of three relations: the effect of the body center(the first and the second term), 
the effect of the base limb(the third term), and the effect of other limbs(the last term). 
The CoM Jacobian with fully specified embedded motions J fsem is a (3x^) matrix where n x 
is the dimension of the base limb, which is smaller than that of the original CoM Jacobian, 
thus the calculation time can be reduced. After solving Eq. (16), the desired joint motion of 
the base limb is obtained. The resulting base limb motion makes a humanoid robot balanced 
automatically during the movement of the all other limbs. With the desired joint motion of 
base limb, the desired joint motions of all other limbs can be obtained by Eq. (7) as follow: 

qi,d = J t(Kd + x ii J Ad)> 

for i = 2,---,n. The resulting motion follows the given desired motions, regardless of 
balancing motion by base limb. In other words, the suggested kinematic resolution method 
of CoM Jacobian with embedded motion offers the WBC(whole body coordination) function 
to the humanoid robot automatically. 

3.3 CoM Jacobian with partially specified embedded motion 

In some cases, the desired motion of any limb is specified in the joint configuration space. 
For example, let us consider that the walking motions (for the leg limbs of 1=1,2) are 
partially specified in Cartesian space and the other limb motions (for z=3,4) in joint space, 
then the kinematic resolution method of Eq. (14) in the previous section should be slightly 
modified as follows: 



J 



psem 

J,„ 



psem, d 



^2,d J 2 \ X 2,d + ^21*A#1, d) 



(18) 
(19) 



where the CoM desired motion and CoM Jacobian with partially specified embedded 
motion are expressed by, respectively, 



C psem,d C d J c^ 2 X 2,d 2-i cfli,d 



(20) 



J = -J + r X i +J + J r J?X 7 ,J, . 

psem v, c\ co-, c, c^ 2 21 1 
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The walking motion is generally specified as the Cartesian desired motions for dual legs but 
the motions of other limbs can be specified as either joint or Cartesian desired motion. Also, 
if we are to implement the robot dancing expressed by desired joint motions, then the 
desired dancing arm motions as well as desired CoM motion are embedded in the suggested 
resolution method and then the motion of base limb (support leg) are automatically 
generated with the function of auto-balancing. This is main advantage of the proposed 
method. 



4. Design of balancing controller 

Since a humanoid robot is an electro-mechanical system including many electric motors, 
gears and link mechanisms, there are many disturbances in implementing the desired 
motions of CoM and ZMP for a real bipedal robot system. To show the robustness of the 
controller to be suggested against disturbances, we apply the following stability theory to a 
bipedal robot control system. The control system is said to be disturbance input-to-state 
stable (ISS) (Choi & Chung, 2004), if there exists a smooth positive definite radially 
unbounded function V(e,t), a class K x function y x and a class K function y 2 such that 
the following dissipativity inequality is satisfied: 



V<-y x {\e\) + y 2 (\s\\ 



(21) 



where V represents the total derivative for Lyapunov function, e is the error state vector 
and s is disturbance input vector. In this section, we propose the balancing 
(posture/ walking) controller for bipedal robot systems as shown in Fig. 4. 
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Fig. 4. Balancing Controller for Humanoid Robot 

In this figure, first, the ZMP Planer and CoM Planer generate the desired trajectories 
satisfying the following differential equation: 



■ llG)2 nkd f° r i = *,y- 



(22) 



Second, the simplified model for the real bipedal walking robot has the following dynamics: 
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c i = u t + s i (23) 

Pi ~ c i ~ l/^n^- f or i = x >y 
where s. is the disturbance input produced by actual control error, u i is the control input, 
c. and p i are the actual positions of CoM and ZMP measured from the real bipedal robot, 
respectively. Actually, the real bipedal robot offers the ZMP information from force/ torque 
sensors attached to the ankles of humanoid and the CoM information from the encoder data 
attached to the motor driving axes, respectively, as shown in Fig. 4. Here, we assume that 
the disturbance produced by control error is bounded and its differentiation is also 
bounded, namely, | s. \< a and | s. \< b with any positive constants a and b. Also, we should 
notice that the control error always exists in real robot systems and its magnitude depends 
on the performance of embedded local joint servos. The following theorem proves the 
stability of the balancing controller to be suggested for the simplified bipedal robot model. 
Theorem 1 Let us define the ZMP and CoM error for the simplified bipedal robot control system 
(23) as follows: 

e P ,i=Pi, d -Pi 

e ci= c i, d - c i f° r i = *,y. 
If the balancing control input u t in Fig. 4 has the following form: 

u i=c?- k P ,i e P,i +k ci e c,i ( 24 ) 

under the following gain conditions: 



k ci > co n and < k . < 



— 7 



(25) 



with any positive constants satisfying the following conditions: 



H-P 2 

< co n and y < / 



then the balancing controller gives the disturbance iwput(s i ,s i )-to-state{e pi ,e ci ) stability (ISS) to a 

simplified bipedal robot, where, the k p . is the proportional gain of ZMP controller and k c . is that of 

CoM controller in Fig. 4. 

Proof. First, we get the error dynamics from Eq. (22) and (23) as follows: 

e c ,i = ^ 2 n (e CJ -e pJ ). (26) 

Second, another error dynamics is obtained by using Eq. (23) and (24) as follows: 

k P .i e P,i = Ki + k c,i e ci + ^ ( 27 ) 

also, this equation can be rearranged for e c : 
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Ki =k P,i e P ,i- k c,i e c,i- £ r 



(28) 



Third, by differentiating the equation (27) and by using equations (26) and (28), we get the 
following: 



e pi = \lk p . (e ci + k ci e ci + s i ) 



col -hi 



V p> 1 J 



a n - k P A, 



(29) 



„,, 7 v , Cfl ,; 



Fourth, let us consider the following Lyapunov function: 



^(e«,..e,,.) = |[(*«,< " ^X- + KA\ 



(30) 



where V(e c ,e p ) is the positive definite function for k . > and £ c t > co n , except e c . = and 
e . = . Now, let us differentiate the above Lyapunov function 



= -k ik 2 . - co 2 )e 2 . - k„ Acq 2 - k„ ; k .)e 2 „ , - (k 2 . - a> 2 )e r . .s. + k„ ; e„ .£. - k„ ; k ; e„ .s. 

c,i\ c,i ns c,i p,i^ n p,i c,is p,i V c,i ns c,i i p,i p,i i p,i c,i p,i i 



-KM, -*iK- KM - KAMI, + K - rt) 
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2a 



1 2 

+ T £ i 

4a 2 l 
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(31) 



where e 2 c . term is negative definite with any positive constant satisfying a < *Jco n and e pi 

term is negative definite under the given conditions (25). Here, since the inequality (31) 
follows the ISS property (21), we concludes that the proposed balancing controller gives the 
disturbance input(£.,£. ) -to-state (e pi ,e ci ) stability (ISS) to the simplified control system 

model of bipedal robot. □ 
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The balancing control input u t suggested in above Theorem 1 is applied to the term c d in 
Eq. (17) or Eq. (20). In other words, two equations (17) and (20) should be modified to 
include the balancing controller (24) to compensate the CoM/ZMP errors as follows: 

<W = (P d ~k p e p +k c e c )- Yj J-x l4 (32) 



psem,d 



= (c d -k p e p +k c e c )- J J + 2 x 2J - |V q id (33) 



where k v and k c imply the diagonal matrices with the corresponding k p j and k C/ i elements, 
respectively, and e v and e c imply the corresponding vectors with e Vl i and e c ,i elements, 
respectively. Also, in order to obtain the desired joint driving velocity, the kinematic 
resolution method of Eq. (16) or Eq. (18) suggested in previous section should be applied to 
the real bipedal robot. 

Remark 2 Note that the ZMP controller in above theorem has the negative feedback differently from 
the conventional controller. Also, for practical use, the gain conditions of balancing controller can be 
simply rewritten without arbitrary positive constants p and y as follows: 

k ci > co n and < k pi < co n for i = x,y, 

because the stability proof is very conservative in above theorem. 

The suggested balancing control method can be divided as the kinematic resolution and 
closed-loop kinematic control of CoM Jacobian with embedded motion. First, the proposed 
kinematic resolution method has main advantage such that it offers the whole body 
coordination function such as balance control to humanoid robot automatically. Second, the 
proposed closed-loop kinematic control method offers the stability and robustness to 
humanoid motion control system against unknown disturbances. For arbitrary given arm 
motions such as dancing, the partially specified embedded CoM motion term c d is 

automatically changed with the desired arm motions ( q 3 d and q 4d ) in Eq. (33), and then 

both desired leg motions ( q x d and q 2d ) are generated by using the equations (18) and (19). 

Like this, the suggested kinematic resolution method offers the whole body coordination 
function such as balance control to humanoid robot automatically. 

5. Experimental results 

In this section, we show the performance of proposed kinematic resolution method and the 
robustness of balancing controller through experiments for humanoid robot v Mahru I 1 
developed by KIST. Its Denavit-Hartenberg parameters for kinematics and centroid/mass 
data for CoM kinematics are in Table 1. These data are utilized to resolve the CoM Jacobian 
with embedded motions kinematically. 

First, in order to show the automatic balancing (or the function of WBC) by the kinematic 
resolution method developed in section 3, we implemented the dancing motion of 
humanoid robot. The desired dancing arm motions shown in Fig. 5 are applied to the dual 
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Left Leg 
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Table 1. The DH parameters, centroid and mass data of KIST humanoid v Mahru I' 
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arms, and then the motions of support left/ right legs are generated as shown in Fig. 6 by the 
suggested kinematic resolution method of CoM Jacobian with embedded dancing arm 
motions. Actually, we utilized the kinematic resolution method of CoM Jacobian with the 
partially specified embedded motion, namely, Eq. (18) and Eq. (19). The initial positions of 
CoM are c x = 0.034[m], c y = 0.0[m], c z = 0.6$7[m], respectively. The experimental results 

show the good performance of proposed method. Though the joint configurations of dual 
arms are rapidly changed with the dancing motion given as shown in Fig. 5, the position of 
CoM is not nearly changed at the initial position as shown in Fig. 6. The joint configurations 
of both legs are automatically generated as shown in Fig. 7 to maintain the CoM position 
constantly. Also, we can see in Fig. 8 that the ZMP has the small changes within the bounds 
of ±0.0 l[m] approximately. As a result, we could succeed in implementing the fast dancing 
motion stably thanks to the developed kinematic resolution method of CoM Jacobian with 
embedded (dancing) motion. 
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Fig. 5. Desired Joint Trajectories of Left and Right Arms while Dancing 
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Fig. 7. Experimental Result: Actual Joint Trajectories of Left and Right Legs generated by the 
developed Kinematic Resolution Method while Dancing 
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30 40 50 60 
Time [s] 

Fig. 8. Experimental Result: Actual ZMP Trajectories while Dancing 

Second, in order to show the performance of the suggested balancing controller of Eq. (33) 
against external disturbances, we realized the corresponding experiments only for both legs 
as shown in Fig. 9. In experiment, if we push the robot forward or pull it backward, then the 
ZMP errors are caused and these ZMP errors give rise to the balancing controller input of 
Eq. (33) and then the balancing control input is resolved using the suggested resolution 
method of Eq. (18) and Eq. (19). Hence, the robot was able to recover the original posture as 
shown in Fig. 9. These results demonstrate the robustness and performance of proposed 
balancing controller. 




Fig. 9. Experimental Result: Performance of Balancing Controller 
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6. Concluding remarks 

In this chapter, the kinematic resolution method of CoM Jacobian with embedded task 
motion and the balancing control method were proposed for humanoid robot. The proposed 
kinematic resolution method with CoM Jacobian offers the whole body coordination 
function to the humanoid robot automatically. Also, the disturbance input-to-state stability 
(ISS) of the proposed balancing controller was proved to show the robustness against 
disturbances. Finally, we showed the effectiveness of the proposed methods through 
experiments. 
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9. Appendix 

9.1 Proof of Eq. (5) 

In Fig. 1, the position of z-th limb represented on the world coordinate is given by 

r=r +R°r . 

1 O O I 

where R is the rotation matrix of body center frame with respect to world coordinate 
frame. Let's differentiate above equation, then 

r,=t+ K*R° r ,+ Rfr, <- [a x]b= -b[ x a] 
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in which, 



[ax]-- 



—a z a y 
a z -a x 
-a v a r 



Now, if we include the angular velocity, then the total velocity of z-th limb motion 
represented on the world coordinate can be obtained as follows: 

r i = r o -[ R ° r i x \ co o +R o r i 

0) t = co o + R o °G) i 
Therefore, 



r 


-[*>,*]" 


K 


+ 


IX 


0,1 


°r. 


>3 


h J 


_«»_ 


L°3 


^J 


°co i 



in short, 



^xrX + XfJa 



9.2 Proof of Eq. (8) 

In Fig. 1, the position of CoM represented on the world coordinate is given by 

c=r +tR °c, 

i = \ 

Let us differentiate above equation, then 

c = t + i( A o c l +R °c l ) 

i=\ 

i=\ i=\ 

n 

c = r + co x ( c - r ) + Vi? °c. <— °c. =° J q 

o o V o / / -i o ; z cz 1 1 

i=l 

••• c = r o + ^ x ( C - r o ) + YR °JA 



9.3 Derivation of CoM jacobian of /-h limb: V c 

The CoM position of k-th link in z-th limb represented on the body center frame is given by 



352 



Advances in Robotics, Automation and Control 



- i,k ' jfc-1 ' *\fc-l ^ jfc 



where °r k _ l and °^_j mean the position and rotation matrix of (/c-2)-th link frame in z'~th 
limb represented on the body center frame, respectively, and 



l Ck= R Mk)°k 



cos(q k ) sm(q k ) 

sin(^) cos(q k ) 

1 



in which R z (q k ) is the z-directional rotation matrix about k-th link driving axis and a k 

means the constant centroid position of k-th link with respect to (k-l)-th frame, this should 
be obtained from the design procedures of robot similarly to the Denavit-Hartenberg 
parameters. Let us differentiate and rearrange above equation like this: 



'k-\ ^ ^Jfc-l L k^ ^k-\ L k 






since °a>. =° co,_. + "R, 



: l,+°<9 t X°«(ft^» 



in which 



°^k =0 ^k-i + 4k(° z k-i) 



°^k-2 +0 ^k-l X (° r k 



2 )- 



where ° z k _ x means the z-direction (or driving axis) vector of (k-l)-th link frame represented 
on body center frame. For instance, for k=l (first link of z-th limb), 

°c it i=° r +°a\x°R R 2 (q l )cr 1 

= q l [°z x°R R z (q l )a l l 

for k=2 (second link of z-th limb), 

°c i2 =° r l +°co 2 x°R l R(q 2 )a 2 

= q l [°z x( o r- o r )] + [q l ( o z ) + q 2 ( o z l )]x°R l R z (q 2 )cr 2 
= q 1 [ o z xCr- o r 0+ o R 1 R z (q 2 )a 2 )] + q 2 [ o z 1 x°R 1 R z (q 2 )a 2 ] 
for k=3 (third link of z-th limb), 

°c u ,=°r 1 + co^R 1 R z {q i )a, 

=9 l rz.x(v 2 -v )]+? 2 rz l x(v r v l )]+[? 1 rz )+^rz 1 )+^cz 2 )]x^( ?3 )(T 3 

= ^rz x(V 2 -V +^^( ? 3)a3)] + ^rz 1 x(V 2 -V 1 +^^( ft )o-3)] + ^rz 2 x^J? z ( ? 3)a 3 ] 
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for k = n j (last link of i-th limb), 

°c,„, =°^-i+X,xX,-A(^)o-„, 

The CoM position of z-th limb represented on the body center frame is obtained as follow: 

o V^ o 

c i = Lvi,k c i,k 

k=\ 

where ju t k means the mass influence coefficient of k-th link in z-th limb defined as Eq. (10). 
Also, its derivative has the following form: 



Therefore, 



where 



: Z H,k 



Z [jpJ 2 ,J3,--,J n . 



k=\ 



J 2= !«, f * ■* C r t _- ° r ,+ °R h _A (g k )a „)} 



h=LM,A' z M'r k - 1 -'r 2 +'R t _ l R,(q t )<T l )} 



j..=n*.?*..-i*( K-A(i..)<r..)} 



fn short, 



o • o 



c> =OJ A 
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In addition, the total CoM position represented on the body center frame is also obtained as 
follow: 
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Multiresolutional Filter Application for Spatial 
Information Fusion in Robot Navigation 

Ozer Ciftcioglu 
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The Netherlands 

1. Introduction 

Robot navigation is one of the major fields of study in autonomous robotics (Oriolio, G., 
Ulivi, G. et al, 1998; Beetz, M., Arbuckle, T. et al, 2001; Wang, M. & Liu, J.N.K., 2004). In the 
present work, application of a multiresolutional filter for spatial information fusion in vision 
robot navigation is described. The novelty of the research is the enhanced estimation of the 
spatial sensory information in autonomous robotics by means of wavelet decomposition 
into multiresolutional levels and the fusion of the processed information while the 
processing takes place at the respective levels. Although wavelet-based information fusion 
is used in different applications (Hong, L., 1994; Hsin, H.C. & Li, A.C., 2006), its application 
in robotics is not common in literature. A wavelet-based filtering application in perceptual 
robotics is articulated earlier where human perception was central to the research. In the 
present work, optimal vision signal estimation for an autonomous robot is presented, where 
vision information is obtained by appropriate shaping of the visual sensory information and 
optimal estimation is obtained by a multiresolutional filter. One of the peculiarities of the 
research is essentially the wavelet-based dynamic filtering rather than static wavelet 
decomposition for image analysis or off-line decomposition for signal analysis. A 
multiresolutional dynamic filtering is a rather complex system. In this system vector form of 
wavelet decomposition is required as briefly mentioned by Hong (Hong, L., 1993) where the 
description of a multi-sensor fusion rather than wavelet decomposition was central to the 
study. In contrast to (Hong, L., 1993), in this work the multiresolutional dynamic filtering is 
central to the study together with the emphasis on application peculiarities in autonomous 
robotics; namely, several options for vector-wise operations are pointed out and one of the 
options is described in detail. Also, in autonomous robotics, the estimation of angular 
velocity is not a measurable quantity and it has to be estimated from the measurable state 
variables so that obstacle avoidance problem is taken care of. Therefore, the angular velocity 
estimation in real-time is a critical task in autonomous robotics and from this viewpoint the 
multiresolutional spatial information fusion process is desirable for enhanced robot 
performance. For the fusion process extended Kalman filtering can be used and is actually 
used in the present study. Perceptual vision employed in this study is one of the vision 
alternatives in robotics providing several essential possibilities characterizing the robot as to 
its performance. Especially perceptual robotics is gaining importance in the last decade as to 
its expected connection to human-like behaviour. The organization of the work is as follows. 
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After brief description of wavelet transform, a detailed description of vector wavelet 
decomposition and inverse wavelet transform is presented with a vector decomposition and 
reconstruction example. The optimal processing of decomposed information is briefly given 
as proposed by Hong (Hong, L., 1993). This is followed by fusion of information from 
different multiresolutional levels that it is based on minimum fusion error covariance. 
Finally, autonomous robot implementation is described with the experimental results 
illustrating the sensory information and estimated trajectory thereby demonstrating the 
effective navigation of a moving robot. The work is accomplished in a virtual environment 
with the intention of realization in real-life environment in due course as the work 
constitutes a common place for diverse robotics areas like autonomous, perceptual, vision, 
cognitive etc. 

2. Perceptual vision and multiresolutional decomposition 

In this work a robot with perceptual vision is considered. The robot is provided with vision 
via sensors which can measure the distance between the robot and the forward direction via 
rays interacting with the environment. The rays can be conveniently shaped within a vision 
cone with a narrow solid cone angle. This is illustrated in figure 1. 




Fig. 1. Perception measurement in the virtual reality. Perception is determined as probability 
by means of the rays impinging on an object. 

Vision rays interact with the environment which includes side walls and hindrance along 
the forward direction. For an environment a permanent fictive vertical planes at certain 
threshold distances are considered. If a distance is more than the threshold value it is taken 
to be the threshold value. This means if the real environment is in front of that fictive plane 
the actual distance measured by the sensor is considered as distance. If the measured 
distances are beyond the predetermined threshold distances the robot moves forward 
without any angular velocity. The position estimation of robot is accomplished by means of 
Kalman filtering via the states of the robot navigation dynamics as presented earlier 
(Ciftcioglu et al., 2007). In this work, the enhanced estimation of the states is investigated via 
wavelet transform (Mallat, S.G., 1989; Ogden, T., 1997; Mallat, S., 1999; Percival, D.B. & 
Walden, A.T., 2000). 
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2.1 Wavelets 

The novel feature of wavelets is that they are localized in time and frequency as to signals. 
This behaviour makes them convenient for the analysis of non-stationary signals. It is an 
elementary introduction of wavelets by introducing a scaling function, such that 

<p{t) = jTZ d h t <p(2t-k) (1) 

k 
A counterpart of this function is called mother wavelet function obtained from 

yr(t) = j2ZgM2t-k) (2) 

k 

where Ik and hk are related via the equation 

&=("!)* V. (3) 

In signal processing literature (3) is known as the quadrature mirror relation and the filter h 
and g as quadrature mirror filters. The coefficients gk and hk that is, the quadrature mirror 
filters are used to compute the wavelet transform. (j)(t) and y/ (t) form orthogonal functional 
low-pass and high-pass filters respectively which are spaces in LiiSf) where inner product of 
functions with finite energy is defined. The orthogonal spaces satisfy the property 

(P m ,{t)yjy/{t) mk =cp m+lk {t) (4) 

where 

<p mJc (t) = 2- l2 <p(2-t-k) (5) 

Wmk (t) = T ll V {Tt-k) (6) 

ra=0 constitutes the coarsest scale. The simplest filter coefficients are known as Haar filter 
and given by 

^ (7) 

Haar wavelets have generally practical value for several reasons. The building blocks in 
decomposition are discontinuous functions that are not effective approximating smooth 
functions. However, because of their very simple form, especially in real-time measurement 
applications their value is eminent. The present application constitutes such an application 
and demonstrates their effectiveness. More about wavelets can be found in the literature 
(Mallat, S.G., 1989; Ogden, T., 1997; Mallat, S., 1999; Vidakovic, B., 1999; Percival, D.B. & 
Walden, A.T., 2000). 

2.2 The Kalman filter 

Kalman filter is an estimator of states of a dynamic system with a minimal error variance 
and in this sense it is optimal. The dynamic system is given in a form which is 
terminologically referred to as state-space: 
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x(k + 1) = A(k)x(k) + B(k)w(k) (8) 

z(k) = C(k)x(k) + v(k) (9) 

where A is the system matrix, B process noise matrix, C is the measurement matrix. Further, 
w(k) and v(k) are Gaussian process noise and measurement noise respectively with the 
properties 

E{w(k)} = 

E{w(k) w(l) T } = Q(k)for k = I ( 10 ) 

= otherwise 
In this work, Kalman filter is used to combine the information from the measurements at 
different resolutional levels and enhance the state estimation rather than to employ single 
measurement at each time-step. The estimation by Kalman filter is accomplished 
recursively. As to the conventional notations used in the literature (Anderson, B.D.O. & 
Moore, J.B., 1979; Maybeck, P.S., 1979; Brown, R.G. & Hwang, Y.C., 1997; Zarchan, P. & 
Musoff, H., 2005; Shalom, Y.B., Li, X.R. et al, 2006; Simon, D., 2006), the recursion is carried 
out by standard matrix equations given below. 

x(k + l\k) = A(k)x(k\k) (12) 

P(k + 1 1 k) = A(k)P(k | k)A(k) T + B(k)Q(k)B(k) T (13) 

K ( k + i) = P(k + i\ k )c (k+ iy 

c(k + i)P(k + 1 1 k)c(k + iy + R(k + 1) 

So that the updated state variables and covariance matrix are 

jc(£ + 1|£ + 1) = jc(£ + 1|£) + 

(15) 
([z(k + l)-C(k + l)x(k + l\k)]K(k + l) K } 

P(k + \\k + \) = [I-K(k + l)C(k + l)]P(k + 1 1 k) (16) 

2.3 The Multiresolutional filter 

N-level multiresolutional dynamic system can be described by 

x l »\k N+ l) = A™(k N )^(k N ), 

z v] (k ) = C ln (k )x v] (k ) + v l ' ] (k ) (17) 

i = l,...,N 
where i=N is the highest resolution level, so that 

E[^(k N )] = 0, 

E[^(k N W N, dJ] = Q m (k N ), k = l (18) 

= k*l 
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Referring to the measurements z( shalom ' YB ' u > XR - et al )(ki) at different resolution levels, we 
write 



and 



E[w [i] (t)] = 0, 

E [ w [i] (t ) w m (/. y ] = Q [i] (t X k = l 
= k±l 

E[v™(k)] = 0, 

E [v [i] (t )v [i] (/. ) T ] = R [i] (t ), k = l 

= k*l 



(19) 



(20) 



Measurements at different resolution levels can be carried out in different ways. One way of 
doing this is shown with the following scheme. 



o 



o •/ 



o 



3 4 5 



4 1=1 



7 1=2 



6 T\ 8 9 10 11 1=3 



where different resolution levels are indicated by 1=1,2,3 while the highest resolution level is 
for 1=3. In this implementation a moving window is used. When samples and 1 are 
available in the window at 1=3, sample is calculated at the resolution level 1=1 by wavelet 
decomposition; when the samples 1 and 2 are available at 1=3, the sample 1 is calculated at 
the resolution level 1=2 which is followed by the calculated sample at the level 1=1 using 
the samples and 1 at the level 1=2. In this way the moving window multiresolutional 
dynamic filtering can be executed. 

Another alternative for the multiresolutional measurements can be accomplished by the 
following scheme. 



i=l 



data block 


data block 


data block i 





i 


2 


1 


2 3 


4 5 


12 3 


4 5 6 7 
time 


8 9 10 11\ 



i=2 



i=3 



In this scheme each data block at the highest resolution level (i=3) contains 4 samples and at 
this very level, the Kalman filtering estimations within the data block are carried out using 
the last data sample of the preceding block, together with the respective data samples within 
the block. Namely, for instance using the data sample 3 at the level i=3, 1-step ahead, 2-step 
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ahead, 3-step ahead and 4-step ahead Kalman predictions are made and as the data samples 
4, 5,6, and 7 are available, the respective estimations are made in sequence at the time points 
where the measurements are available; that is at the time points 4, 5, 6, and 7. 
Another scheme of measurements at different resolution levels are shown in figure 2 and the 
wavelet decomposition of state variables in a data block is shown in figure 3. This is the 
actual scheme implemented in this work and therefore explained in details as follows. It is 
to note that propagation as to state estimation occurs block-wise at the highest resolution 
level. Decomposition of the states by wavelet transform and reconstruction by inverse 
wavelet transform is effective in this scheme due to better estimation of the error covariance 
as to the state estimation as explained below. 



data block 


data block 


data block 
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i=l 
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4 5 6 7 
time 


TT7^// 
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Fig. 2. Measurements at different resolution levels 
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Fig. 3. Wavelet decomposition of state variables in a data block 

Within a data block, the state variables at resolution level ;' are designated as 



X w = 



x [i] k(i) 




Jt m jfc(l+1) 




jt m Jfc(/ + 2*- 


l )_ 



(21) 
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x [i, k(i) - 



(22) 



where m is data block index and s is the number of state variables. In a data block, there are 
2 11 state variables. Each state variable has s state components. A lower resolution state 
variable is computed from 



x [i] (t ) = h x x {m (k M ) + h 2 x [i+l] (k M + 1) 



(23) 



where hi and hi are the Haar low-pass filter coefficients. The details component i.e., high 
frequency part after the decomposition is computed via 



y l '\k i ) = g l x lM \k i J + g 2 x^(k M +l) 



(24) 



where gi and gi are the Haar high-pass filter coefficients. The reconstruction of the states is 
carried out by combining (22) and (23) in a matrix equation form as given below. 



v-t'+l] 



(*m) 



x [WI (£,„+!). 



hWykJ + g'^ik,) 



(25) 



Wavelet decomposition and reconstruction is carried out according to the scheme shown in 
figures 4 and 5. 
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Fig. 4. Wavelet decomposition of state variables in a data block 
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Fig. 5. Wavelet reconstruction of state variables in a data block 
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The wavelet matrix operator G and the scaling matrix operator H in the decomposition and 
their counterparts G* and H* in the reconstruction contain two-tap Haar filters and they are 
related by 

G* = G T H* = H T (26) 

It is to note that the state variable estimations are carried out at the respective resolution 
levels, as follows. 

H'^' = diag{{\ I V2)/*<" , (1 / V2)Af ,...., (1 / fijk™ } (27) 

where K is the number of filters involved in a decomposition from the resolution level i+1 to 
i. For instance from 2 to 1, then i=l, and K is given by 

K = 2 M = 1 (28) 

as this is seen in figure 2, as the block length is 2. For the s state variable of the system, the H 
matrix is composed of s Haar filters at the diagonal as 

H = diag{H m ^ -,// [2] w - , ,H lK r'} (29) 

Similarly, for the reconstruction filter, we write 

G M - =diag{(j2)gl\(j2)g?,....,(j2)g^} (30) 

G = diagiG^' ,G [2 ^' , ,G m +1 -'} (31) 

As K is given by (27). For the inverse transform scheme given by figure 5, we write 

H'=diag{H m " M ,H li r i > > H ^ M ) (32) 

G* = diag{G^ M ,G [2 ~ M , ,G m '-n (33) 

where 

H- M =diag{(^2)hr,(^2)hr,....,(^2)h: m } (34) 

G>^=diag{(V^g?\(l/j2) g y,....,(l/J?)g T r} (35) 

Above T indicates transpose and K is given by (28). 

2.3.1 Example block-wise 2-way multiresolutional computation 

An example block wise wavelet computation is presented in this subsection. For this we 
consider figure 2 and the initial data block designated as with the data samples as xo and 
xi at the resolution level z=2. Also for the sake of simplicity we consider s=2; namely only 
two state variables of the system. Therefore the data samples are denoted as xoi, X02 and xn, 
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X12; first index is for the data sample, second index for the state variable. We define an 
auxiliary matrix L as follows 



n 2, = 



1 


0" 


1 





1 








1 



(36) 



which yields 



L m X m = 



10 0" 


~x [2] ~ 




~x [2] ~ 


10 


x [2] 

A 02 




x [2] 


10 


x [2] 




x [2] 


1_ 


x [2] 

_ 12 




x [2] 

_ 12 



(37) 



where m is index of the data block which is ra=0, above. It is to note that, the sate variables 
are grouped slightly different for computational convenience; namely the decomposition is 
carried out by 



x [l - 



-[i-l] _ tTI 



L T[l - ll HL [ll X [l 



where Y m are the wavelet coefficients. From the definition given by (27) and (30) 
H i+l ^ = diag{{\ I V2)^ (1) } = [1 / 2 1/2] 

G»^=diag{(j2)g?} = [l -1] 
Also from (29) and (31) 

G = diag{G m +l ^,G [l] i+l ^} 
H = diag{H m ^\H [l ^} 

In (34) substituting z=2, the decomposition at the level z=l, becomes 



(38) 
(39) 

(40) 
(41) 

(42) 
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.JX 01 +.JX U 



(43) 



which is a low-pass filtered data block by averaging. The wavelet coefficients which are the 
high-pass filtered data samples are obtained in a similar way as 
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Y? ] =L T[1] GL [2] X [2] = 



1 
1 
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~x [2] " 
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(44) 



(43) and (44) are the multiresolutional decomposition of the data block 0, from level z=2 to 

the level z=l. 

The inverse wavelet transform is carried out according to the general scheme given by figure 5. 



From (32) and (33), we write 
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(45) 



(46) 



so that, (45) becomes 



CT=diag{G™,G m M } = 



x m = jjm H > L m x m +L nn G > jmyin 
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The substitution of the respective matrices into (48) yields 
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(47) 



(48) 



(49) 
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which is X m P] as given in (37). 

For the same decomposition from z=3 to z=2, the auxiliary matrix L is given by 



1 

10 

10 

10 

10 

10 
10 

J) 1 

For this case z'=2, and K is given by 

K = 2 M = 2 
as this is seen in figure 2, where the block length is 4. 



(50) 



(51) 



2.3.2 Estimate updating 

The basic update scheme for dynamic multiresolutional filtering is shown in Fig. 6 where at 
each resolutional level, when the measurement is available, the state variables are updated 
and when the block is complete the inverse wavelet transform and fusion is performed. 
During the inverse transformation wavelet coefficients Y m +i| m +iW saved aside are used. 
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Fig. 6. Wavelet decomposition of state variables in a data block 
Explicitly 
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-* W+ W1 -A m+ l\m +K m + \ \ L m+\ ~ C m + l A m + \\m ) 



and 



P [i] 

± XXm+\\m+\ 



■■(I-Kj , C,J , )P xi 



(53) 



(54) 



As X m +i | m M and Y m +i | m H, Y m +i | m [ i+1 ] are correlated, the covariance matrices PxYm+i | m [i] and 
Pyxm+iimM are also updated. However, the wavelet coefficients Y m +i| m W and their covariance 
matrices PYYm+i|m [i] are not updated. The minimum variance Kalman gain matrix K m +iW at 
each level, is determined by 



Y M _ p \.i\p [iV( r [i]p [i]p [i]T n [i]\ 

^m+X ~ r XXm+\\m ^ ' m+\ \^m+l £ XXm+\\m ^ m+\ " t " ^Sw+l ) 

where the measurent matrix C m =iM and Rm+it 1 ] are given by 



c !Ii = diag 



Ci = dia s 



C m [(m + l)2 M ],C m [(m + l)2 M -2 M +1], ...,' 
,C m [(m + l) + 2 M +l] 

^ m [(m + l)2 M ] ? ^ m [(m + l)2 M -2 M +1], ...," 
,i? m [(m + l) + 2 M +l] 



(55) 



(56) 



(57) 



Once, within the moving window, the sequences of updated state variables and error 
IA and P m+llm J N,l] for z=2,2,..,N are determined, they must be fused to 



covariances X 



generate an optimal X m J NF] and X m 
minimum fusion error covariance X .. ^ [NF] 



X^r and X m ^r ■ For the 



is calculated as 



, the fused estimate X 



V INF] _ p [NF} 

■^ m+\\m+\ m+l\m+l 



1ft- ™) x * 

_ '=1 
where the minimum fusion error covariance P ^ 



j (N-\){p m ^rYx^r (58) 

} NF] becomes 



( p [nf] v 1 = y (p [nj] v 1 -(N-\)(p lN] V 1 



(59) 



The fused estimate X 



[NF] 



is a weighted summation of both predicted X m+l]m and 
updated X m+lh+l [N ' l] , for i=l,2,..,N. The sum of the weight factors equal to the identity I. This 
can be seen by substitution of P m +l[m J NF] given above into the expression of X m+l[m J NF] in 
(58). 

3. Autonomous robot navigation 

The computer experiments have been carried out with the simulated robot navigation. The 
state variables vector is given by (22) while taking i=N. 
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x m k(N) = 



(60) 



Explicitly, 



x [ ^(N) = [x,x 9 y,y,co] 



where oe> is the angular rate and it is estimated during the move. When the robot moves in a 
straight line, the angular rate becomes zero. The other state variables are x and y coordinates 
and the respective velocities. 

The experiments are carried out as follows. For a pre-defined trajectory, perception 
measurements along the trajectory are obtained in a virtual reality environment. This 
trajectory is designated as reference. The measurements as data samples are used in 
sequence for a real-time robot navigation by multiresolutional dynamic filter estimation. 
Designing the experiment in this way allows one to assess the effectiveness of the robot 
navigation by comparison of the actual navigation path with the estimated one. The overall 
robot trajectory is shown in figure 7 where there are three lines plotted but only two of them 
are visible. The line marked by * sign represents the measurement data set. The line marked 
by • is the multiresolutional dynamic filter estimation. The line indicated by o sign is the 
reference trajectory. These lines are not explicitly seen in the figure. For explicit illustration 
of the experimental outcomes the same figure with a different zooming ranges and the 
zooming powers are given in figures 8-11. 
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Fig. 7. The overall Robot trajectory with measurement (* signs), and multiresolutinal 
dynamic filter estimation (• signs). 
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trajectory ref [r], MDF [b] and measurements [g] 
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Fig. 8. Enlarged robot trajectory, measurement, and multiresolutinal dynamic filter 
estimation in angular velocity mode. The * sign is for measurement, o sign for the reference 
and • sign is the estimated trajectory. 

From the experiments it is seen that, the multiresolutional filtering is effective for estimation 
of the trajectory from perception measurement. Estimations are more accurate in the 
straight-ahead mode seen in figure 11, compared to the cases where angular velocity does 
not vanish. This can be explained by noting that in the straight-ahead mode the angular 
velocity is zero and the system matrix is fixed. However in the non-linear mode the system 
matrix is approximate due to the estimation error of the angular velocity. In this case the 
slight difference between the reference trajectory and the estimated trajectory is markedly 
seen due to the approximation error caused by the Taylor's series expansion and ensuing 
linearization in (extended) Kalman filtering. 
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Fig. 9. Enlarged Robot trajectory, measurement multiresolutinal dynamic filter in angular 
velocity mode. The * sign is for measurement, o sign for the reference and • sign is the 
estimated trajectory. 
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trajectory ref [r], MDF [b] and measurements [g] 
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Fig. 10. Enlarged Robot trajectory, measurement multiresolutinal dynamic filter in angular 
velocity mode. The * sign is for measurement, o sign for the reference and • sign is the 
estimated trajectory. 
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Fig. 11. Enlarged robot trajectory, measurement multiresolutinal dynamic filter straight- 
ahead mode. The * sign is for measurement, o sign for the reference and • sign is the 
estimated trajectory. 



4. Discussion and conclusion 

The detailed description of processing of spatial information for a perceptual robot is 
described. Especially, the perception is considered to be a probabilistic concept and defined 
as such (Ciftcioglu, O., Bittermann, M.S. et al, 2006) so that the probabilistic consideration in 
perceptual robot is desirable for human-like robot navigation. This facilitates the robotic 
movement coupling the cognition of a robot with the environment. In this context, Bayesian 
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perception and cognition studies (Knill, D.C. & Richards, W., 1996) can be more 
substantiated with more accurate priors. This is especially important when one deals with a 
certain scene but from a different vantage point. 

In vision robotics autonomous navigation is a challenging issue. Using the perceptual 
spatial information vision robot can navigate autonomously by perception-based sensory 
information so that it can react better with respect to the environmental and circumstantial 
situations. Better is in the sense of obstacle avoidance and way finding without crash. In the 
present work, the processing the spatial information is accomplished by means of wavelet 
technology that is the main component in the information fusion process. The simplest form 
of wavelets is used to cope with the real-time requirements in the navigation and it is 
demonstrated that, this choice is justified. Kalman filtering is the machinery for the fusion 
process, as well as it plays the role of optimal observer (Friedland, B., 1986) of the 
measurements referring to distance (r), angle (6) and the scalar velocity (v). These may 
partly be computed from the estimated state variables, in the case of unavailable actual 
measurement counterparts. For that matter, it is the angular velocity for which the 
measurements are unavailable, in this work. The work is a clear demonstration of the 
working state observers and therefore an important hint for the physical robotics exercises. 
It is reminded that, the present work is an autonomous robotics simulation implemented in 
a virtual reality and the results reported are the computer experiments. 
Applications of vision robot are countless and they are well documented in the literature. 
Autonomous robotics with perception is a new dimension in a vision robot as such its vision 
can be shaped for application dependent vision properties. The investigations presented in 
this work are motivated by measuring the perception aspects of an architectural design from 
different vantage points and altogether to obtain a compromise among them, as to a final 
decision-making for that design. Such a task in one hand is a routine work for a robot in 
virtual reality and the actual (real) scene having been represented in the virtual reality, the 
scene can be perceived in many different ways as desired. The same task for a human is in 
the first place extremely tedious but more importantly subjective in terms of perception 
measurement and therefore can be deemed to be inconclusive due to probable 
inconsistencies among different observers and even for a single viewer. This is because of 
the complexity of the task due to the number of objects involved in the scene. Here the 
objects may be a number of dwelling units subject to marketing by a building project 
developer. Since a number of locations and architectural requirements impose condition on 
perception aspects, the task is a multi-objective optimization subject to these impositions as 
constraints. Also, in this situation one is interested in a solution on a Pareto front. Such 
complexities are today effectively dealt with thanks to evolutionary algorithms (Deb, K., 
2001; Coello, C.A.C., Veldhuizen, D.A. et al, 2003; Sato, H., Aguirre, H.E. et al, 2007). Some 
endeavours in the context of architecture are reported in (Bittermann, M. et al., 2008; 
Ciftcioglu, O. & Bittermann, M.S., 2008). The present research is another endeavour along 
this line revealing the exhaustive details of spatial information fusion as to accurate robot 
navigation using multiresolutional dynamic filter approach. 
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1. Introduction 

The problem of pursuit and evasion is a classic problem that has intrigued mathematicians 
for many generations. Suppose a target or evader is moving along a given curve in the 
plane. A pursuer or chaser is moving such that its line of sight is always pointing towards 
the target. The classic problem is to determine the trajectory of the pursuer such that it 
eventually captures the target. There is extensive literature on this problem, see for example 
the recent book by Nahin (Nahin, 2007). A special case of the pursuit problem is the case 
where the curve of the target is a circle. This is a classic problem which was first treated by 
Hathaway (Hathaway, 1921), see also the book by Davis (Davis, 1962). In the classic 
problems, the line of sight is always pointing towards the target. This method of pursuit is 
also known as 'pure pursuit' or 'dog pursuit' or 'courbe de chien' in French. Although there 
have been some modern works on extensions of the classic problem of pursuit, see for 
example (Marshall, 2005), here we completely abandon the 'pure pursuit' restriction and we 
treat the problem where the pursuer is required to capture or intercept the target in a given 
prescribed time or in minimum time. Also, in the classic problem, the velocities of the target 
and pursuer are assumed constant and there is no consideration of the physics of the 
motion, such as thrust forces, hydrodynamic or aerodynamic drag and other forces that 
might be acting on the target and pursuer. 

In an active-passive rendezvous problem between two vehicles, the passive or target vehicle 
moves passively along its trajectory. The active or chaser vehicle is controlled or guided 
such as to meet the passive vehicle at a later time, matching both the location and the 
velocity of the target vehicle. An interception problem is similar to the rendezvous problem, 
except that there is no need to match the final velocities of the two vehicles. On the other 
hand, in a cooperative rendezvous problem, the two vehicles are active and maneuver such 
as to meet at a later time, at the same location with the same velocity. The two vehicles start 
the motion from different initial locations and might have different initial velocities. 
An optimal control problem consists of finding the control histories, i.e., the controls as a 
function of time, and the state variables of the dynamical system such as to minimize a 
performance index. The differential equations of motion of the vehicles are treated as 
dynamical constraints. One possible approach to the solution of the rendezvous problem is 
to formulate it as an optimal control problem in which one is seeking the controls such as to 
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minimize the differences between the final locations and final velocities of the vehicles in 
some mathematical sense, for example in the least squares sense. Here we use a minimax 
formulation in order to define the objective function that contains the terminal constraints. 
The methods of approach for solving optimal control problems include the classical indirect 
methods and the more recent direct methods. The indirect methods are based on the 
calculus of variations and its extension to the maximum principle of Pontryagin, which is 
based on a Hamiltonian formulation. These methods use necessary first order conditions for 
an optimum, they introduce adjoint variables and require the solution of a two-point 
boundary value problem (TPBVP) for the state and adjoint variables. Usually, the state 
variables are subjected to initial conditions and the adjoint variables to terminal or final 
conditions. TPBVPs are much more difficult to solve than initial value problems (IVP). For 
this reason, direct methods of solution have been developed which avoid completely the 
Hamiltonian formulation. For example, a possible approach is to reformulate the optimal 
control problem as a nonlinear programming (NLP) problem by direct transcription of the 
dynamical equations at prescribed discrete points or collocation points. 
Direct Collocation Nonlinear Programming (DCNLP) is a numerical method that has been 
used to solve optimal control problems. This method uses a transcription of the continuous 
equations of motion into a finite number of nonlinear equality constraints, which are 
satisfied at fixed collocation points. This method was originally developed by Dickmanns 
and Well (Dickmanns, 1975) and used by Hargraves and Paris (Hargraves, 1987) to solve 
several atmospheric trajectory optimization problems. Another class of direct methods is 
based on biologically inspired methods of optimization. These include evolutionary 
methods such as genetic algorithms (Goldberg, 1989), particle swarm optimization methods 
and ant colony optimization algorithms. 

Genetic algorithms (GAs) provide a powerful alternative method for solving optimal control 
problems. They have been used to solve control problems (Crispin, 2006, 2007), orbital 
transfer and rendezvous problems (Crispin and Ricour, 2007). GAs use a stochastic search 
method and are robust when compared to gradient methods. They are based on a directed 
random search which can explore a large region of the design space without conducting an 
exhaustive search. This increases the probability of finding a global optimum solution to the 
problem. They can handle continuous or discontinuous variables since they use binary 
coding. They require only values of the objective function but no values of the derivatives. 
However, GAs do not guarantee convergence to the global optimum. If the algorithm 
converges too fast, the probability of exploring some regions of the design space will 
decrease. Methods have been developed for preventing the algorithm from converging to a 
local optimum. These include fitness scaling, increased probability of mutation, redefinition 
of the fitness function and other methods that can help maintain the diversity of the 
population during the genetic search. 

2. Interception and rendezvous as optimal control problems 

We study interception and rendezvous problems for vehicles moving in an incompressible 
viscous fluid such as water. The vehicle has a propulsion system that delivers a thrust of 
constant magnitude T and is controlled by varying the thrust direction, i.e., thrust vectoring. 
Since the fluid is viscous, a drag force acts on the vehicle, in the opposite direction of the 
velocity. The motion takes place in a horizontal plane, the (x, z) plane, either at the surface of 
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the water or at a constant depth. The target vehicle is moving along a circle of radius R. We 
describe the motion in a cartesian frame of reference (x, z) with its origin at the center of the 
circle, with x positive to the right and z positive upwards. 

Let the angle ft denote the orientation of the thrust vector T, which is also the angle of the 
velocity vector V. The angle /?(t), which depends on the time t, is measured positive counter- 
clockwise from a reference horizontal line, the positive direction of the x axis. In the 
examples we are going to present, the motion of the chaser will start from a point located 
outside the circle and will be moving from right to left, see Figure 2. In this case, it is 
convenient to use the complementary angle y (t)= jt - /?(t) as the control. 

The interception problem is an optimal control problem, in which it is required to determine 
the control function or control history y(t) of the chaser vehicle, such that it will meet the 
target vehicle at a prescribed location at the terminal time tf . Since GAs deal with discrete 
variables, we discretize the values of y (t). The motion of the vehicle is governed by Newton's 
second law of motion and the kinematic relations between velocity and distance: 

mdV/dt = T + D (1) 

dx/dt = Vcos/3(t) (2) 

dz/dt = Vsin/3(t) (3) 

where /?(t)= n - y(t) is the angle of velocity vector V, T is the thrust and D is the drag force 
acting on the body. 

dV/dt = T/m + D/m (4) 

Writing this equation for the components of the forces along the tangent to the vehicle's 
path, we get: 

dV/dt = T/m - D/m (5) 

Here V, T and D are the magnitudes of the velocity, thrust and drag vectors, respectively. 
We introduce the drag coefficient C D \ 

D = \ P V*SC D (6) 

where p is the fluid density, S is a typical cross-section area of the vehicle. The coefficient of 
drag depends on the Reynolds number Re = p Vd/ji where d is a typical length dimension of 
the vehicle, e.g., d = \fs. 

Substituting the drag from (6) and writing T = amg, where a is the thrust to weight ratio 
T/mg, equation (5) becomes: 

dV/dt = ag- pV 2 SC D /2m (7) 

Introducing a characteristic length L c , time t c and speed V c as 
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(8) 



L c = 2m/pSC D , t c = ^L c /g, v c = ^fgL~ c 
the following non-dimensional variables, denoted by a bar, can be defined: 

t = {L c /gfin, V = (gL c )V 2 7 (9 ) 

Substituting in (7), we have: 

dV/dt = a -V 2 (10) 

Similarly, the other equations of motion can be written in non-dimensional form as 

dx/dt = — Vco^{t) / n \ 

d~zfdt = Vsin7(£) / 12 ) 

For each vehicle the initial conditions are: 

V(0) = V 0t x(0)=x 0> z(0) = z (13) 

We now define a rendezvous problem between two vehicles. We denote the variables of the 
first vehicle by a subscript 1 and those of the second vehicle by a subscript 2. We will now 
drop the bar notation indicating non-dimensional variables. The two vehicles might have 
different thrust to weight ratios, which we denote by a\ and «2, respectively. The equations 
of motion for the system of two vehicles are: 

dV x jdt = ai - V? 

dx\/dt= — FiCOS7i(t) 

dzi/dt = Visinrfi(t) 

(14) 
dV 2 /dt = a 2 - V 2 2 

dx 2 /dt= -V 2 cosj 2 (t) 

dz 2 /dt = V2sin7 2 (£) 

We consider the case where the second vehicle, which is the target vehicle, is constrained to 
move along a circular trajectory. In this case, the magnitude of the velocity vector V 2 is 
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constant. The angular speed U), the azimuth angle 9(t) and the required control angle 
72 {t) are given by: 

lo = V 2 /R, 0(t) = ut, l2 {t)=n/2-6{t) (15) 

where R is the radius of the circle. 9 it) is measured at the center of the circle, positive 
counter-clockwise, from the positive x direction. The vehicles can start the motion from 
different locations and at different speeds. The initial conditions are given by: 

V 1 (Q) = V 10 , xi(0)=xio, zi(0) = zio 

V 2 (0) = V 2 o, x 2 (0)=x 20 , z 2 (0) = z 20 ( 16 ) 

The rendezvous problem consists of finding the control function ji(t) such that the two 
vehicles arrive at a same terminal location on the circle and at the same speed in the given 
time. The terminal constraints are given by: 

xi(t f ) =x 2 (t f ) 9 zi(t f ) = z 2 (t f ), V 1 (tf) = V 2 (t f ) (17) 

In order to fulfill the terminal constraints using a genetic algorithm, we define the following 
objective function: 

f[ Xl (t f ), x 2 (t f ), Zl (t f ), z 2 (t f ), V,{t f ), V 2 (t f )] = 
= minmax{(a;i(* / ) - x 2 {t f )f, ( Zl (t f ) - z 2 {t { ))\ (V lx (t f ) - V 2x (t f )) 2 , 

(V lz (t f )-V 2z (t f )f] (18) 

where Vix{tf)i ^2ar(^/)» Viz{tf) and ^(t/) are the x and z components of the velocity 
vectors of the two vehicles at the terminal time t = tf m We can also define an interception 
problem, of the target-chaser type, in which the target vehicle is passive and the chaser 
vehicle maneuvers such as to match the location of the target vehicle, but not its velocity. 
Consistent with the above terminal constraints, we define the following objective function 
for the interception problem: 



f[xi{t f \ x 2 (t f ), Zl {t f \ z 2 {t f )] = mMmax{(x 1 {tf)-x 2 (tf)) 2 ,{z 1 {t f )-z 2 {t f )f} 



(19) 



We have also tried an objective function defined by the distance between the two terminal 
points of the two vehicles, but it was found that the mini-max objective function works 
better in satisfying the terminal constraints. We use standard numerical methods for 
integrating the differential equations. The time interval is divided into N time steps of 
duration At = tj/N. The discrete time is U = iAt. We used a second-order Runge-Kutta 
method with fixed time step. We also tried a fourth-order Runge-Kutta method and a 
variable time step and found that the results were not sensitive to the method of integration. 
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The control function j(t) is discretized to 7(2) = j(U) according to the number of time 
steps N used for the numerical integration. Depending on the accuracy of the desired 
solution, we can choose the number of bits n 1 for encoding the value of the control j(i) at 
each time step i. The number of bits used for encoding 7(2) and the number of time steps N 
will have an influence on the computational time. Therefore n 1 and N must be chosen 
carefully in order to obtain an accurate enough solution in a reasonable time. We use 
smoothing of the control function by fitting a third-order polynomial to the discrete values 
of 7(0 . The values of the polynomial at the N discrete time points are used as the current 
values of 7(0 and are used in the integration of the differential equations. 
An appropriate range for 7(2) is 7 6 [ — tt, tt]. We choose N=30 or 40 as a reasonable 
number of time steps. We now need to choose the parameters associated with the Genetic 
Algorithm. First, we select the lengths of the "genes" for encoding the discrete values of 7(0 . 
A choice of rf = 8 bits for Vz e [0, TV — 1] wa s made. A reasonable size for the population 
of solutions is typically in the range n pop ^ [50, 200] p Q r this problem, there is no need for a 
particularly large population, so we select n pop = 50 # The probability of mutation is set to a 
value of 5 percent £Wt — 0.05. 

3. Chaser-target interception 

We present an example of a chaser-target interception problem between two vehicles. The 
motion takes place in the horizontal plane (x, z). The first vehicle is active (vehicle 1) and the 
second vehicle is the passive or target vehicle (vehicle 2) and moves at a constant speed V 2 
along a circular trajectory. The target vehicle starts from a point on the circle and keeps 
moving along its circular trajectory. The chaser vehicle starts at a point outside the circle and 
the interception occurs at a point on the circle at the final time. The interception point is not 
known a priori. The parameters and initial conditions for this example are given below. 



to = 0, tf = 35, a\ = 4a/9, a2 = a = 0.05, 71 G [ — 7r, 7r] 
The initial conditions are: 



(20) 



x 1 (0) = 4, zi(0) = 0, 2:2(0) = 1, z 2 (0) = 

Vi (0) = ^/E[ = 0.2236, V 2 (0) = JT 2 = 0.1491 (21) 

Since this is an interception problem, we do not require matching between the final 
velocities. The objective function is defined by equation (19). The parameters of the genetic 
algorithm are summarized in Table 1. 
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Table 1. Parameters for the interception problem with t/= 35 
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The results of this example are displayed in Figures (1-4). Figure 1 displays the control 
function of the chaser vehicle as a function of time for a terminal time of 35 units. 




Fig. 1. Control function for the chaser vehicle in a chaser-target interception on a circle with 
a terminal time of 35 units. 




Fig. 2. Trajectories for a chaser-target interception on a circle with a terminal time of 35 
units. 
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Fig. 3. Trajectories x(t) for a chaser-target interception on a circle with a terminal time of 35 
units. 



1.5 




Fig. 4. Trajectories z(t) for a chaser-target interception on a circle with a terminal time of 35 
units. 

The trajectories of the target and chaser vehicles are shown in Figure 2. The target starts the 
motion at the point (1,0) on the circle and the chaser starts at the point (4,0) outside the 
circle. The line with dots is the trajectory of the target. The line with circles is the trajectory 
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of the chaser. The continuous curve without dots marks the circle. Figures 3 and 4 show the 
trajectories of the target and the chaser vehicles in parametric form, with the time as a 
parameter. Figure 3 displays the x(t) coordinates of the two vehicles as a function of time 
and Figure 4 displays the z(t) coordinates for the two vehicles. The lines with dots are the 
target trajectories and the lines with circles are the chaser trajectories. It can be seen that the 
final boundary conditions are fulfilled, that is, the coordinates of the two vehicles are equal 
at the final time. 

In the above example, a non-dimensional terminal time of 35 units was used. We now look 
at the problem of finding the minimum time for interception for the same set of parameters 
and initial conditions as in the previous example. We decreased the final time to 30 units 
and then to 25 units and were able to obtain a solution, i.e., the terminal constraints are 
fulfilled for the shorter times. We then tried to decrease the final time to a value of 23 units 
but we could not obtain a solution that fulfills the terminal constraints. We then tried a 
slightly higher value of 24 units and were able to obtain a solution. We therefore conclude 
that the minimum time for interception in this case is very close to 24 units . Since we have 
shorter times, we decreased the number of discrete time points from N=40 to N=30. The 
results for this example are given in Figures (5-8). Figure 5 shows the control function of the 
chaser vehicle. Comparing with Figure 1, we can see that the control looks much different 
because of the shorter final time. Figure 6 shows that the interception occurs as a head-on 
collision, a phenomenon that occurs also in the classic problem of pursuit. From Figure 7, it 
can be seen that the horizontal distance between the chaser and the interception point is 
reduced linearly at maximum speed, because of the minimum final time. In this case, it can 
also be seen from Figures 7 and 8, that the final boundary conditions are fulfilled. 




Fig. 5. Control function for the chaser vehicle in a chaser-target interception on a circle with 
a minimum terminal time of 24 units. 
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Fig. 6. Trajectories for a chaser- tar get interception on a circle with a minimum terminal time 
of 24 units. 
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Fig. 7. Trajectories x(t) for a chaser-target interception on a circle with a minimum terminal 
time of 24 units. 
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Fig. 8. Trajectories z(t) for a chaser-target interception on a circle with a minimum terminal 
time of 24 units. 

4. Rendezvous between two vehicles 

We treat a rendezvous problem between two vehicles where one vehicle is moving along a 
circular trajectory. The first vehicle starts from a point outside the circle. The second vehicle 
starts at a point on the circle. The final time is given and the rendezvous point can occur at 
any point on the circle. The vehicles have the same thrust to weight ratio a. We present 
results with a final time of 25 units, which is close to the minimum time. 



t = 0, t f = 25, a x =a = 0.05, a 2 = a = 0.05, ^ G [ 
The initial conditions are: 



7T, 7T 



(22) 



H (0) = 4, zi(0) = 0, a; 2 (0) = 1, z 2 (0) = 0, V[(0) = yfiT u V 2 (0) = ^T 2 



(23) 



The parameters for this test case are summarized in Table 2. The results are given in Figure 
9. The curves with dots show the trajectories of the target and the curves with circles show 
the trajectories of the chaser vehicle. The upper left part of the figure shows the trajectories 
in the plane (x, z). The chaser vehicle is maneuvering such as to approach the rendezvous 
point from behind, in order to match the velocity vector of the target vehicle. A similar effect 
can be seen in Figure 10, which displays results similar to Figure 9, but with the chaser 
vehicle starting from rest. It is interesting to note that in this case, the z coordinate of the 
chaser, follows the z coordinate of the target, as can be seen in the bottom right of Figure 10. 
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Table 2. Parameters for the rendezvous problem 
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Fig. 9. Rendezvous between two vehicles on the circle when the outer vehicle starts at an 
initial speed. Top left: Trajectories of the two vehicles. Top right: Control thrust angle of the 
vehicle starting outside the circle. Bottom left: Horizontal coordinates as a function of time. 
Bottom right: vertical coordinates as a function of time. 
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Fig. 10. Rendezvous between two vehicles on the circle when the outer vehicle starts from 
rest. Top left: Trajectories of the two vehicles. Top right: Control thrust angle of the vehicle 
starting outside the circle. Bottom left: Horizontal coordinates as a function of time. Bottom 
right: vertical coordinates as a function of time. 
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5. Conclusion 

The interception and rendezvous problems between two autonomous vehicles moving in an 
underwater environment has been treated using an optimal control formulation with 
terminal constraints. The vehicles have a constant thrust propulsion system and use the 
direction of the thrust vector for steering and control. We use a genetic algorithm to 
determine directly the control history of the vehicle by evolving populations of possible 
solutions of initial value problems. In order to fulfill the final boundary conditions as 
terminal constraints, a mini-max objective function has been defined. An interception 
problem, where one vehicle moves along a circular trajectory at constant speed and the 
second vehicle acts as a chaser, maneuvering such as to capture the target in a prescribed 
time has been solved. The problem of minimum time to interception has also been treated. 
The rendezvous problem where the target vehicle moves along a circle and the chaser 
vehicle starts from a point outside the circle either from rest or with an initial velocity has 
also been solved for a terminal time close to the minimum time for rendezvous. This method 
can be extended to include multiple autonomous vehicles. Another direction for future 
research is to include additional realistic effects besides thrust and drag. For example, 
additional effects can be included to better describe the dynamics of the vehicles, such as 
finite size, rigid body dynamics, inertia and added mass effects. 
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1. Introduction 

Traditional methods of data collection are often expensive and time consuming. We propose 
a novel data collection technique, called Bayesian Adaptive Sampling (BAS), which enables 
us to capture maximum information from minimal sample size. In this technique, the 
information available at any given point is used to direct future data collection from 
locations that are likely to provide the most useful observations in terms of gaining the most 
accuracy in the estimation of quantities of interest. We apply this approach to the problem of 
estimating the amount of carbon sequestered by trees. Data may be collected by an 
autonomous helicopter with onboard instrumentation and computing capability, which 
after taking measurements, would then analyze currently available data and determine the 
next best informative location at which a measurement should be taken. We quantify the 
errors in estimation and work towards achieving maximal information from minimal 
sample sizes. We conclude by presenting experimental results that suggest our approach 
towards biomass estimation is more accurate and efficient as compared to random 
sampling. 

Bayesian Adaptive Sampling (BAS) is a methodology that allows a system to examine 
currently available data in order to determine new locations at which to take new readings. 
This procedure leads to the identification of locations where new observations are likely to 
yield the most information about a process, thus minimizing the required data that must be 
collected. As an example of the application of this methodology, we examine the question of 
standing woods in the United States. In order to estimate the amount of carbon sequestered by 
trees in the United States, the amount of standing woods must be estimated with quantifiable 
uncertainty (Wheeler, 2006). Such estimates come from either satellite images or near ground 
measurements. The amounts of error in the estimates from these two approaches are currently 
unknown. To this end, an autonomous helicopter with differential GPS (Global Positioning 
System), LIDAR (Light Detection and Ranging), stereo imagers, and spectrometers has been 
developed as a testing platform for conducting further studies (Wheeler, 2006). These 
instruments are capable of measuring the reflectance data and the location of the Sun and 
helicopter in terms of the zenith and the azimuth angles (Figure 1). The objective is to develop 
a controlling software system for this robotic helicopter, which optimizes the required ground 
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sampling. There are a number of methods by which data maybe collected. The first simplistic 
data collection method is to conduct an exhaustive ground sampling, that is, to send the 
helicopter to every possible location. The second approach is to perform random sampling 
until the estimates have acceptable standard errors. Although random sampling presents a 
possibility that the helicopter will take samples from the locations that offer the greatest 
amount of information, and therefore reduce the needed sample size, there is no guarantee 
that such a sample set will be chosen every time. The third and more efficient method is to take 
only a few samples from "key" locations that are expected to offer the greatest amount of 
information. The focus of this paper is to develop a methodology that will identify such key 
locations from which the helicopter should gather data. 

O 




Fig. 1. Viewpoint and Position of the Sun. s , <p s are the zenith and the azimuth angles of the 

Sun, and 6 v , (j) v are the zenith and the azimuth angles of the view, respectively (Wheeler, 

2006). 

In the work described here, the key locations are identified using current and previously 
collected data. The software works in tandem with the sampling hardware to control the 
helicopter's position. Once a sample has been obtained, the data are fed into the system, 
which then calculates the next best location to gather further data. Initially, the system 
assumes an empirical model for the ground being examined. With each addition of data 
from the instruments, the parameter estimates of the model are updated, and the BAS 
methodology is used to calculate the helicopter's next position. This process is repeated until 
the estimated uncertainties of the parameters are within a satisfactory range. This method 
allows the system to be adaptive during the sampling process and ensures adequate ground 
coverage. 

The application employs a bi-directional reflectance distribution function (BRDF), in which 
the calculation of the amount of reflection is based on the observed reflectance values of the 
object, and the positions of the Sun and the viewer (Nicodemus, 1970). The advantage of 
using this function is that it enables the system to compensate for different positions of the 
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Sun during sampling. Once the reflectance parameters are estimated, BAS uses the principle 
of maximum entropy to identify the next location where new observations are likely to yield 
the most information. 

In summary, the BAS methodology allows the system to examine currently available data 
in order to determine new locations at which to take new reflectance readings. This 
procedure leads to the identification of locations where new observations are likely to yield 
the most information. 

2. Background and related work 

Computing view points based on maximum entropy using prior information has been 
demonstrated by Arbel & Ferrie (1970), where this technique was used to create entropy 
maps for object recognition. Vazquez et al. (2001) also demonstrated a technique for 
computing good viewpoints based on Information Theory. Whaite & Ferrie (1994) 
developed an autonomous explorer that seeks out those locations that give maximum 
information without using a priori knowledge of the environment. Makay (1992) used 
Shannon's entropy to obtain optimal sample points that would yield maximum information. 
The sample points are taken from the locations that have largest error bars on the 
interpolation function. In our work, the optimal locations that offer the maximum amount of 
information are identified using the principle of maximum entropy, where the maximization 
is performed using techniques suggested by Sebastiani and Wynn (2000). 
A simplified but elucidating version of an example given in Sebastiani and Wynn (2000) is 
as follows: Suppose that the relationship between a dependent variable y and an 
independent variable x is known to be linear over the domain of interest x e [1,4], and is 
known to be one of two equations with additive noise, yi = 5 + lx\ + £i or y\ = 17 + 3x/ + z\, 
each with a Bayesian prior belief to update of 1/2. Figure 2 shows the two possibilities over 



' - y = 5 + 7x 
- - - y = 1 7 + 3x 



,-*-' 




Fig. 2. Two lines, with circle indicating the region where the two models have the greatest 
diference, and thus where the maximum entropy sampling procedure indicates a new 
observation should be taken in order to best determine which model is more likely correct. 
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the domain of interest. Our intuition tells us that an observation taken at x = 3 will not help 
us to discriminate between the two possible models, as the expected resonse, E( y \ x=3) is 
the same for each of the two models. Our intuition further tells us that an observation taken 
at x = 1 will yield a response, y \ x=l, providing the most "information" as to the correct 
model, since the two models are most distinct at x = 1. An application of the theorems of 
Sebastiani and Wynn show that the optimal location of the independent variable at which to 
take an observation that maximizes the entropy (expected information as defined by 
Shannon (1948)) is at x* = arg x min{(5 + 7x) - (17 + 3x)} 2 = arg x min{144 - 96x + 16x 2 } =1 , which 
agrees with our intuition. 

3. Application and model 

Surface BRDF is often used to measure vegitation or other attributes of a surface. By it's very 
nature, BRDF requires measurements taken from a variety of viewing angles and sun 
positions (and hence different times of day), as solar radiation reflected by the surface is not 
uniform in all directions (Zhang et al., 1997), nor is it uniform from the same direction when 
the sun is shining on the location from different directions, as can been seen in Figure 3. This 
shows the importance of including not only the viewpoint position but also the position of 
the sun when taking reflectance measurements in order to accurately measure ground cover. 
The model for the data used in our framework is based on the semi-empirical MISR (multi- 
angle imaging spectrometer) BRDF Rahman model (Rahman et al., 1993): 

r(0 s ,0 v ,&,<P v )=P [cos(0, ) cos(#„ ) {cos(0, ) + cos(#„ )}] *"' 
xexp(-b-p(n))h(0 s ,e v J s J v ) 

where 

G(0,,0 v ,^J = Vtan 2 (^ (3) 

p(D) = cos(<9 ) cos(# ) + sin(<9 ) sin(<9 ) cos(^ - (f) v ) (4) 

where r = r(6 s ,@ v ,<fi s ,<fi v )is the measured reflectance, p is the surface reflectance at 
zenith, k is the surface slope of reflectance, b is a constant associated with the hotspot, or 
"antisolar point" (the point of maximum reflectivity, which is the position where the sensor 
is in direct alignment between the Sun and the ground target), S , </> s are the zenith and the 

azimuth angles of the Sun, respectively (Fig. 1), and 6 V , <fi v are the zenith and the azimuth 
angles of the view, respectively (Fig. 1), where in each case the zenith angle 6 ranges from 
to nil (horizontal to vertical) and the azimuth angle ranges from to In (all the way 
around a circle), thus the viewpoint (and the sun) are assumed on a half-hemisphere 
centered over the object or in this case region of land to be studied. 
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With this application, the goal of this study is to estimate the parameters p , k and b to a 
predefined level of accuracy, or as some in the engineering field say, to "minimize the 
length of the error bars of the paramater estimates", with as few observations as is 
reasonably possible. 




Fig. 3. The effect of forward scattering of light (on the left) and backward scaterring of light 
(on the right) in two photographs of the same location taken from the same position at 
different times of day (Lucht & Schaaf, 2006). 



4. Methodology 

Our framework consists of the following two steps: 

1. Parameter Estimation: In this step, we estimate the values of the parameters (p , k and 
b ), and their covariance matrix and standard errors, given currently available data of 
the amount of observed reflected light, and the zenith and azimuth angles of the Sun 
and the observer. 

2. Bayesian Adaptive Sampling (Optimal Location Identification): In this step, we use the 
principle of maximum entropy to identify the key locations from which to collect the 
data. 

Once the key location is identified, the helicopter goes to that location, and the instruments 
on the helicopter measure the reflectance information. These data are then fed into the 
Parameter Estimation stage and the new values of the parameters ( p , k and b ) are 



392 



Advances in Robotics, Automation and Control 



calculated. This process is repeated (Fig. 4) until the standard errors of the parameter 
estimates achieve some predefined small value, ensuring adequacy of the estimated 
parameters. This process minimizes the required number of observations to be taken by 
ensuring that each new observation is taken from a location which maximizes the expected 
information obtained from the forthcoming observation, and using the then observed data 
in order to determine the location from which to take the subsequent observation. 



e s? e v? (j> s? <j> Vj r 




Parameter 
Estimation 



p ? k ? b Biomass 



i 



Bayesian Adaptive 
Algorithm 



Estimation 



R-\a 



Fig. 4. Overview of Bayesian Adaptive Sampling. 

5. Implementation 

5.1 Parameter estimation 

The input to this step is the observed reflectance value (r ), zenith and azimuth angles of the 
Sun(# 5 ,^ 5 ), and zenith and azimuth angles of the observer/helicopter(# v ,^ v ) . The 

parameters ( p , k , and b ) are estimated using the following iterated linear regression 
algorithm: Taking the natural logarithm of r = r(6 s ,0 v ,<fi s ,<fi v ) results in the following near 
linear version of this model: 



lnr = In/? + (k - l)ln[cos(0Jcos(0 v ){cos(0J + cos(<9 v )}] 



(5) 



Note that aside from the term ]nh(0 s ,0 v ,0 s ,0 v ) , which contains a nonlinear/? , the function 
ln(r ) is linear in all three parameters, ln(/?) , k , and b . // Linearization ,/ of ln(A) is 
accomplished by using the estimate of p from the previous iteration, where at iteration n in 
the linear least-squares fit the value of h(0 s ,0 v ,0 s ,<fi v ) from (2) is modified and is taken to 
be the constant in (6) where p (0) is set equal to zero. 



!-/><" 



h M (0 s ,& r ,0 s ,0 v ) = l + - 
We thus obtain as out model, for observation i, the linear model 



(6) 
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y t =fi +kx n + bx i2 + s t (7) 

where we assume that the errors in the model, £ i , are independent and identically 
distributed (iid) random variables from a Normal distribution with mean zero and 
variance cr 2 ; fl = \np is the regression constant/ intercept to be estimated; x n x i2 axe the 

independent/ explanatory variables given in (8) and (9), which are functions of the 
viewpoint and the position of the sun as described above and are measured at observation 
time i ; and y. is a function of the response and functions which are known/ fixed constants 
at the given time i as shown in (10). 

x n = ln[cos(0 SI . ) cos(0 w . ){cos(<9 5 . ) + cos(0 w . )}] (8) 

x i2 = -p(Cl)= -[cos(^.)cos(6> v .)+sin(6^Jsin(6> v .)cos(^. -fa)\ (9) 

y . = ln(r. ) + ln[cos(0 w . ) cos(0 w . ) {cos((9 5 . ) + cos(0 w . )}] - In h {n) (0 si , vi , fa , fa ) (10) 

Next, linear regression is performed on the model in (7) (with iteration until the convergence 
of the parameters due to the nonlinear term h(0 s ,0 v ,0 s ,<j> v ) being treated as the 

constant h in) (0 s ,0 v ,<fi s ,<fi v )). From the regression, we find estimates of the following quantities: 

Estimates of the parameters /? , k, and b, an estimate of a 2 (the variance of the errors, as 

discussed in and around (7)), and an estimate of the covariance matrix, R~ l , of the parameter 
estimates, i.e., the estimate of the matrix given in (11). Recall that /? = ln/? so that 
p = exp(/? ) , and through use of the delta method (Casella and Berger, 2002) an estimate 

of pis p=exp(ft) and v(p) - (exp(^ )) 2 v{p Q )=p 2 v(ft). 



R~ l = 



V(/? ) Cov(A,*) Cov(fi 09 b) 
Cov(kJ ) V(k) Cov(k,b) 

Cov(bJ ) Cov(b,k) V(b) 



(11) 



5.2 Bayesian adaptive sampling 

This step identifies the most informative location (0 V ,</> V ) to which to send the helicopter to 
take a new reflectance reading. We employ the principle of maximum entropy, in which the 
available information is analyzed in order to determine a unique epistemic probability 
distribution. The maximization is performed as per techniques suggested by Sebastiani and 
Wynn (2000), where in order to maximize the amount of information about the posterior 
parameters ( p , k and b ), we should maximize the entropy of the distribution function. 

Mathematically, maximizing the entropy is achieved by maximizing In X'E -1 X + i? where 

| A | is the determinent of the matrix A, A' indicates the transpose of the matrix A, E is the 
covariance matrix of the error terms, and X is the regression design matrix where the i th 
row is associated with the i th observation (either a future or past observation, depending on 
the context) 
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X-- 



1 x u 

1 x 2 

1 Xy 



As the natural logorithm is a monotone increasing function, and we have assumed that the 
errors are iid N(0, a 2 ) and so E = — -I where I is the identity matrix , our criteria reduces to 



maximizing the quantity given in (12). 



-X'X + R 



(12) 



Note that a 2 and R are estimated in the Parameter Estimation step and are thus at this 
stage assumed to be known quantities. As described above, the matrix X contains functions 
of the zenith and azimuth angles of the Sun, \0 S , (j) s ) , and the viewpoint/ helicopter, (<9 V ,0 V ) 
at which future observations are to be taken (i.e., for each time i at which future 
observations are to be taken). Since the times at which future observations are to be taken 
are known, and thus the positions of the sun at these times are known, the only remaining 
unknown quantities in (12) are the values of \0 v ,<j) v ). Thus, the new location(s) to which the 

helicopter will be sent are the values of \0 v ,<f> v ) in the rows of X associated with new 
observations that maximize (12). 

5.3 Initialization 

Before data have been collected, one of course cannot calculate parameter estimates as 
described in section 5.1. Before parameter estimates are collected, one does not have an 
estimate of R or a 2 to use in (12) from section 5.2. In order to initialize the procedure, one 
can use a Bayesian prior for these quantities, or equivalently, any estimate based upon prior 
knowledge of the quantities. In the absence of any prior knowledge, one may take an 
uninformative prior for R to be the identity matrix I, and any arbitrary estimate of a 2 . See 
the appendix for a more thourough discussion of the quantity in (12) to be maximized. 



6. Simulation 

We conduct two simulated experiments in which the estimates of the model parameters are 
calculated. In the first experiment, "Estimation Using Random Observations", the data are 
collected by sending the helicopter to random locations. In the second experiment, 
"Estimation using BAS", the data are collected using BAS. 

We note that the Sun moves through In radians in a 24-hour period, i.e., at the rate of 
2n/(24 * 60) « 0.004363323 or slightly less than 0.005 radians per minute. We will assume it 

takes about 2 minutes for the helicopter to move to a new location. Thus, the position of the 
Sun changes by approximately 0.01 radians between measurements. 

In our simulation, the true values of the parameters p , k and b are 0.1, 0.9, and -0.1, 
respectively. For the purpose of this paper, the observed values were simulated with added 
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noise from the process with known parameters. This allows us to measure the efficacy of the 
algorithm in estimating the parameters and minimizing the standard errors of our estimates. 
In actual practice, the parameters would be unknown, and we would have no way of 
knowing how close our estimates are to the truth, that is, if the estimates are as accurate as 
implied by the error bars. 

6.1 Estimation using observations taken at random locations 

In this experiment, we send the helicopter to 20 randomly chosen locations to collect data. 
Starting with the fifth observation, we use the regression-fitting algorithm on the collected 
input data set (the observed reflectance information, and the positions of the Sun and the 
helicopter), to estimate the values of the parameters p , k , b as well as their standard errors. 
Table 1 shows the results of this experiment. 



Obs 


e v 


A 




Estimate (se) 


Estimate (se) 


Estimate (se) 


# 


r 


of p 


of k 


of b 


1 


0.114 


1.673 


0.157552 








2 


0.882 


6.013 


0.156616 








3 


0.761 


0.917 


0.192889 








4 


0.678 


1.308 


0.180404 








5 


0.260 


0.114 


0.152558 


0.0683 (0.1172) 


0.8497 (0.0607) 


-0.5958 (0.1413) 


6 


1.195 


2.367 


0.146659 


0.0767 (0.0932) 


0.7906 (0.0476) 


-0.4506 (0.1040) 


7 


0.237 


2.805 


0.149475 


0.0830 (0.0746) 


0.8268 (0.0404) 


-0.3745 (0.0893) 


8 


0.166 


1.700 


0.155497 


0.0832 (0.0641) 


0.8286 (0.0345) 


-0.3722 (0.0788) 


9 


0.320 


2.012 


0.154191 


0.0831 (0.0572) 


0.8277 (0.0307) 


-0.3735 (0.0713) 


10 


1.224 


4.085 


0.129133 


0.0917 (0.0465) 


0.8369 (0.0381) 


-0.2483 (0.0539) 


11 


1.409 


3.442 


0.135005 


0.0917 (0.0431) 


0.8380 (0.0309) 


-0.2481 (0.0503) 


12 


0.092 


1.559 


0.154096 


0.0920 (0.0394) 


0.8398 (0.0285) 


-0.2462 (0.0471) 


13 


0.806 


0.891 


0.200401 


0.0888 (0.0402) 


0.8129 (0.0284) 


-0.2952 (0.0453) 


14 


1.256 


5.467 


0.147654 


0.0891 (0.0385) 


0.8181 (0.0259) 


-0.2914 (0.0433) 


15 


0.227 


1.284 


0.155373 


0.0889 (0.0368) 


0.8169 (0.0248) 


-0.2919 (0.0418) 


16 


1.129 


5.522 


0.148721 


0.0889 (0.0354) 


0.8174 (0.0236) 


-0.2918 (0.0402) 


17 


0.507 


5.696 


0.150381 


0.0891 (0.0333) 


0.8183 (0.0225) 


-0.2904 (0.0380) 


18 


0.119 


4.363 


0.142232 


0.0890 (0.0302) 


0.8181 (0.0207) 


-0.2908 (0.0357) 


19 


0.245 


0.524 


0.151915 


0.0889 (0.0299) 


0.8172 (0.0205) 


-0.2901 (0.0355) 


20 


0.446 


2.408 


0.144471 


0.0884 (0.0297) 


0.8149 (0.0204) 


-0.2930 (0.0354) 



Table 1. Observations and Estimates Using Random Sampling 

6.2 Estimation using BAS 

In this experiment, the first five locations of the helicopter are chosen simultaneously using 
an uninformative prior distribution (i.e., as no estimate of R has yet been formed; it is taken 
to beer 2 1, with a 2 taken to be 10" 6 based on information regarding the accuracy of the 
instrumentation) and an X matrix with five rows in which the positions of the Sun \0 si , </> si ) 
are known and (12) is maximized over five pairs of helicopter viewpoints (0 vi , (f) vi ) . 

Subsequently, we use BAS to calculate the next single best informative location for the 
helicopter to move to in order to take a new reflectance observation, in which case (12) is 



396 



Advances in Robotics, Automation and Control 



maximized with an X matrix containing a single row in which the position of the Sun 
\9 S , $)is known, a 2 and R are taken to be their estimated values from the regression 
performed on the data collected to this point, and the only unknowns, to be calculated, are a 
single pair of helicopter viewpoint values, (# v ,^ v ) . After the helicopter is sent to this location 
and takes a reflectance reading, a new regression is performed on all data accumulated to 
this point, and the procedure is repeated. In practice, a computer onboard the helicopter 
would perform the regression, calculate the new next-best location, direct the helicopter to 
that location, take the reflectance reading, and repeat until some predetermined stopping 
point, which might include some fixed number of observations, or until the error bars have 
reached some predefined level indicating an acceptable accuracy in the parameter estimates, 
or perhaps until the helicopter has nearly exhausted it's fuel supply. In this simulation, we 
stop at the fixed number of 20 observations. 

Table 2 shows the results from this experiment. In both experiments estimates of the 
parameters, along with their standard errors, cannot be formed until at least five 
observations have been taken. 



Obs 


e v 


I 




Estimate (se) 


Estimate (se) 


Estimate (se) 


# 


r 


of p 


of k 


of b 


1 


0.460 


0.795 


0.172364 








2 


0.470 


0.805 


0.177412 








3 


1.561 


3.957 


0.161359 








4 


1.561 


0.825 


0.183571 








5 


1.265 


3.977 


0.129712 


0.1041 (0.0325) 


0.90904 (0.00879) 


-0.1249 (0.0290) 


6 


0.514 


0.845 


0.173072 


0.1042 (0.0252) 


0.90927 (0.00700) 


-0.1255 (0.0233) 


7 


1.561 


3.400 


0.160130 


0.1045 (0.0223) 


0.90857 (0.00615) 


-0.1220 (0.0199) 


8 


1.172 


4.007 


0.130101 


0.1029 (0.0192) 


0.90547 (0.00577) 


-0.1329 (0.0180) 


9 


0.723 


0.875 


0.189697 


0.1039 (0.0244) 


0.90663 (0.00748) 


-0.1428 (0.0228) 


10 


1.561 


0.885 


0.192543 


0.1042 (0.0213) 


0.90801 (0.00569) 


-0.1394 (0.0185) 


11 


0.527 


0.895 


0.172811 


0.1042 (0.0193) 


0.90796 (0.00523) 


-0.1392 (0.0172) 


12 


1.561 


4.047 


0.164530 


0.1044 (0.0193) 


0.90696 (0.00519) 


-0.1343 (0.0167) 


13 


1.561 


4.057 


0.164822 


0.1046 (0.0190) 


0.90636 (0.00505) 


-0.1314 (0.0161) 


14 


1.137 


4.067 


0.131443 


0.1038 (0.0169) 


0.90483 (0.00471) 


-0.1365 (0.0148) 


15 


0.713 


0.935 


0.183894 


0.1042 (0.0169) 


0.90538 (0.00480) 


-0.1397 (0.0149) 


16 


1.561 


0.945 


0.192280 


0.1048 (0.0163) 


0.90777 (0.00427) 


-0.1333 (0.0136) 


17 


1.187 


4.097 


0.134701 


0.1047 (0.0146) 


0.90757 (0.00399) 


-0.1340 (0.0125) 


18 


0.655 


0.965 


0.176841 


0.1048 (0.0140) 


0.90779 (0.00385) 


-0.1349 (0.0120) 


19 


1.561 


4.117 


0.168819 


0.1049 (0.0142) 


0.90694 (0.00388) 


-0.1321 (0.0120) 


20 


1.148 


4.127 


0.132199 


0.1045 (0.0132) 


0.90617 (0.00373) 


-0.1349 (0.0114) 



Table 2. Observations and Estimates Using BAS 



6.3 Results 

In this section, we compare and analyze the results of our two experiments. The comparison 
results shown graphically in Figures 5-7 show that the estimates using the data from the 
"well chosen" locations using BAS are closer to the true values, p = .1, k = 0.9 and b = -0.1, 
than the estimates based on data from the randomly chosen locations. Also, the error bars 
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using BAS are much shorter, indicating higher confidence in the estimates of the parameters 
based on the "well chosen locations", e.g., the length of the error bar for the estimate 
calculated using data/ observations from five well chosen locations is as short as the error 
bar based on data collected from 20 random locations. 



Using random locations to take observations 

Using locations identified by BAS to take observations 



— r~ 
10 



~r~ 
15 



20 



Number of included observations 
Fig. 5. Estimates and Error Bars for p . 

Within each figure, the horizontal axis indicates the number of observations between five 
and twenty that were used in forming the estimates. The vertical axis is on the scale of the 
parameter being estimated. Above each observation number, an "o" represents the point 
estimate (using the data from the first observation through the observation number under 
consideration) of the parameter using the randomly chosen locations and the observations 
from those locations. The "x" represents the point estimate of the parameter using 
observations taken at locations chosen through BAS. The bars are " error bars" and extend 
one standard error above and below the estimated parameter, based on the data collected at 
locations from the appropriate experiment. The horizontal line represents the true value of 
the parameter in our simulation. 

Note that in Figure 6 and Figure 7, the error bars rarely overlap the true value of the 
parameter. This can be attributed to two factors. In large part, this is due to the fact that they 
are engineering "error bars" with a length of one standard error beyond the point estimate. 
Traditional 95% statistical confidence intervals based on two standard errors would in 
virtually every case overlap the true values. Additionally, these are cumulative plots, in 
which the same data are used, adding observations to form the parameter estimates as one 
moves to the right in each figure. Thus the point estimates and error bars are dependent 
upon one another within a figure, and not independent point estimates and confidence 
intervals. 
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Finally, we see that the estimates using BAS (to select the points from which to take 
observation) are generally closer to the truth than when we use random points to take 
observations, and more importantly the standard errors associated with any given number 
of observations are much smaller. 
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Using random locations to take observations 

Using locations identified by BAS to take observations 
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Fig. 6. Estimates and Error Bars for k . 
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Using random locations to take observations 

Using locations identified by BAS to take observations 
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Fig. 7. Estimates and Error Bars for b . 
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7. Conclusions and future work 

Our initial results have shown that BAS is highly efficient compared to random sampling. 
The rate at which the standard errors, or the error bars, are reduced is much quicker, and 
hence the significant amount of information is found more quickly compared to other 
traditional methods. We have also shown that this methodology performs well even in the 
absence of any preliminary observations. Further simulation has shown evidence that BAS 
can be three times as efficient as random sampling. This efficiency amounts to savings of 
time and money during actual data collection and analysis. 

In addition to the application discussed in this paper, the theoretical framework presented 
here is generic and can be applied directly to other applications, such as, military, medical, 
computer vision, and robotics. 

Our proposed framework is based on the multivariate normal distribution. The immediate 
extensions of this framework will be to accommodate non-normal parameter estimate 
distributions. As part of our future study, we intend to employ sampling methodologies 
using Bayesian Estimation Methods for non-normal parameter estimate distributions. We 
also intend to use cost effectiveness as an additional variable. In this initial work, the focus 
was to identify the viewpoints that would give us the most information. However, it is not 
always feasible or efficient to send the helicopter to this next "best" location. As part of our 
future work, we intend to identify the next "best efficient" location for the helicopter from 
which it should collect data. 
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9. Appendix 

In section 5, we noted that maximizing the entropy of the experiment is equivalent to 

1 



maximizing the quantity 



z X'X + R 



originally given in (12), where R and a come from a 



Bayesian prior belief which in this case could be interpreted as the estimates based upon a 
regression of the data collected up to the given point in time. The matrix X contains a row(s) 
with functions of known position(s) of the sun (6 s , </> s ) and unknown positions of the 



1 Before this research could be implemented in practice, two of the above mentioned liaisons 
left NASA and the then current funding expired. 
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viewpoint/ helicopter (# v ,^ v ) which are then the parameters over which the objective 

function is maximized. In this setting, X is based on future observations. If the X matrix is 
filled with the appropriate functions (as defined in (8) and (9)) from all of the observations 
already taken, each observation being associated with one row in the X matrix, where we 
might call this matrix X obs where the subsript "obs" reminds us that this matrix is based on 
previous observations rather than future observations, then the quantity 



^ obs ^ obs 



(13) 



is known as the observed Fisher Information. In this setting, the observed Fisher 

Information matrix is the inverse of the estimated variance matrix of the parameters 

(Cassella and Berger, 2002; Weisberg, 2005; Montgomery et al, 2006), and thus (13) IS the 

matrix R that is used above after the intial set of observations are taken and estimates can be 

formed. 

Any linear algebra textbook will show that stacking two conformable matricies, say 



X = 



X^ 



X 



will give the result tt'tt = X' obs X obs +X'X and thus, after obtaining intial estimates of the 
variances from the regression and using them for our prior R , and also estimating the error 



variance g 2 , our objective function given in (12) can be written as 



-^X'X + R 

G 



— XX + ^2~ X 'obs X obs 


= 


4 T {x l x+x' obs x obs ) 


G G 




G 



4t) I (** + **** )|=4«-|N'K| 

G J G 

where the third equality follows from the fact that for scalar a and a matrix A of 
dimnension nxn, I aA I = a n \A I , and that X'X + X' ohs X ohs is a 3x3 matrix since we are estimating 

three parameters. Noting that we have no control over g 2 , the estimated variance of the 
error terms, we see that maximizing (12) is equivalent to maximizing I X'tfl , where the first, 
say, n rows of K are based on the functions of si ,0 vi ,<f> si ,<f> vi m (8) and (9) for observations 
which have already been taken, and one final row in which the position of the sun, S ,(/) S , is 
known and the maximization occurs over the two parameters 6 v , (j) v which will indicate the 
new position to which the helicopter should move in order to take the next observation. 2 



2 Of course, this assumes that we are using prior data to define our Bayesian prior for R . If 
we have a strong belief for some other prior distribution of the parameters (and thus a 
different variance matrix R~ l ), then the above will not hold. 
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In summary, we find that maximizing the expected Shannon Information, i.e. the entropy, of 
an experiment, in which the experiment is associated with a regression fit where the errors 
are assumed to be Normally distributed, is equivalent to maximizing the determinent of the 
Fisher Information associated with that experiment. Further study is waranted in cases 
which are not regression based and/ or where errors are not neccessarily Normally 
distributed. 
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1. Introduction 

Data mining (DM) is the process to extract previously unknown and implicit information 
from large databases. Several techniques have been used to discover such kind of 
knowledge; most of them derived from machine learning and statistics. The majority of 
these approaches focus on the discovery of accurate knowledge. However, this knowledge 
should be useless if it does not offer some kind of surprisingness to the final user. Based on 
this idea, some investigations were started recently with the aim of extracting accurate and 
interesting information from datasets. In this sense, data mining can be faced as a multi- 
objective problem. 

Different machine learning approaches have been employed to perform data mining tasks. 
Most of them are based on evolutionary methods, like genetic algorithms (Goldberg, 1989). 
Another advantage of genetic algorithms is that they can be adapted to treat multi-objective 
problems in a Pareto sense. Various multi-objective genetic algorithms have been proposed 
in the literature, like the method known as Non-dominated Sorting Genetic Algorithms (NSGA) 
(Srinivas & Deb, 1994). 

The task performed in a data mining process depending on what kind of knowledge 
someone needs to extract. The main types of tasks performed by DM algorithms are 
classification, association, clustering, regression, sequences analysis, summarizing and 
dependence modelling. Classification task searches for the knowledge able to predict the 
value of a previously defined goal attribute based on other attributes. This knowledge is 
often represented by IF-THEN rules and it is the most investigated data mining task. 
Dependence modelling task can be seen as a generalization of classification. It also aims to 
discover rules able to predict the goal attribute value, from values of the prediction 
attributes. However, in dependence modelling, there are more than one goal attribute. 
Initially, a small set of goal attributes is specified, whose prediction is considered 
interesting. These attributes may occur in the consequent or in the antecedent parts of the 
rule. The others attributes occur only in the antecedent. 

A multi-objective evolutionary data mining environment named MO -miner was 
implemented based on the family of algorithms called non-dominated sorting genetic 
algorithms (NSGA). The two desirable properties of the rules being mined - accuracy and 
interestingness - are simultaneously manipulated. MO-miner keeps the metrics related to 
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these properties separated during the evolution, as different objectives used in the fitness 
calculus in a Pareto-based approach. This chapter describes the investigation performed 
using MO-miner in the mining of accurate and interesting rules for the dependence 
modelling task. MO-miner was tested in two public domain databases named Zoo and 
Nursery. The results obtained by MO-miner had been compared with those generated by a 
standard GA in order to identify the benefits related to the multi-objective approach. 
Section 2 presents a brief review on multi-objective genetic algorithms with emphasis on the 
method used here, the NSGA proposed by Srinivas and Deb (1994). The major data mining 
tasks are presented in Section 3. This section also presents a review of recent works which 
have also investigated the application of multi-objective approaches to data mining tasks. 
Section 4 describes the multi-objective environment named MO-miner, which was 
implemented to perform the dependence modelling task. The two data sets used to evaluate 
the environment are characterized in Section 5. Section 6 presents the results and their 
analysis regarding the application of MO-miner and a comparative single-objective 
environment to the data sets. Finally, Section 7 presents the major conclusions of this work. 

2. Evolutionary algorithms and multi-objective problems 

Evolutionary algorithms (EAs) are computational search methods gleaned from biological 
evolution. In these methods a population of candidate solutions is generated by random 
means, and then evaluated in terms of how close they are to the desired solution of the 
problem at issue. According to the score derived from that evaluation, a selection procedure 
then determines which subpopulation will be used as the basis for the creation of another 
population of candidate solutions. The new population is obtained through the application 
of genetic operators over the selected subpopulation, the expectation being that the new 
population be, in average, better than its predecessor. This process is repeated over 
generations, until a population emerges with a satisfactory candidate solution, or until some 
predefined computational constraint (such as a certain number of generations) is achieved. 
Evolutionary methods enable us to obtain solutions to problems for which there are no 
known exact algorithms or, for those where an algorithm exists, but the solution requires a 
long time of processing to be found (Mitchell, 1997). The genetic algorithm (GA) proposed 
by Holland is the most well-known EA (Goldberg, 1989). The success of GA search is due to 
the symbiosis of three operators: selection, crossover and mutation. 

2.1 Pareto optimum and multi-objective genetic algorithms 

Many real world problems involve simultaneous optimization of multiple objectives, so that 
it is not always possible to achieve an optimum solution in respect to all of them 
individually considered. In this kind of problem, there is a set of solutions better than all the 
other solutions in the search space (Srinivas & Deb, 1994), which is named the Pareto 
optimum or set of non-dominated solutions. 

Suppose that there are N objectives fi, fi, ... /n to be simultaneously optimized. A solution A 
is said to be dominated by another solution B, or B dominates A, if B is better than A in 
relation to at least one of the objectives fi, and is better than or equal to A in relation to all 
other objectives fix . . . fi-i, fi+i ... /n- Two solutions A and B are non-dominated in relation to 
each other if A does not dominate B and B does not dominate A. For example, suppose that 
functions fl and f2 in Figure 1 must be simultaneously maximized. One can affirm that 
solution A is better than solutions C and D; that is, C and D are dominated by A. However, 
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in the case of solutions A and B, it is not possible to affirm which one is the best. Therefore, 
one can say that solutions A and B are non-dominated and they dominate solutions C and D. 

f 2 
3.5 



♦ Dominated 

■ Non-dominated 




fi 







1 



Fig. 1. Example of dominated and non-dominated solutions when maximizing fl and f2. 

The Pareto optimum is the set of non-dominated solutions considering the entire search 
space, that is, any candidate solution that is better than one of the solutions from the Pareto 
optimum in respect to one of the objectives, is guaranteed to be worse with respect to 
another objective. 

Multi-objective evolutionary methods try to find this solution set by using each objective 
separately, without aggregating them as an unique objective (Coello, 1999). Some of the 
most traditional multi-objective evolutionary methods are VEGA (Vector Evaluated Genetic 
Algorithm; Shaffer, 1985), NSGA (Non-dominated Sorting Genetic Algorithms; Srinivas & 
Deb, 1994), MOGA (Multiple Objective Genetic Algorithm; Fonseca & Fleming, 1993), and 
NPGA (Niched Pareto Genetic Algorithm; Horn & Nafpliotis, 1993); the NSGA is briefly 
described below, insofar as it provided the procedures used in the present work. 
More recently, new evolutionary methods have been developed, characterised by the fact 
that - unlike the methods mentioned above - an elitist strategy was incorporated in them, 
leading to methods such as NSGAII (Non-dominated Sorting Genetic Algorithm II; Deb et 
al, 2000), SPEA (Strength Pareto Evolutionary Algorithm; Zitzler & Thiele, 1999), SPEA2 
(Strength Pareto Evolutionary Algorithm; Zitzler et al, 2001), PAES (Pareto Archieved 
Evolution Strategy; Knowles & Cornes, 1999) and PES A (Pareto Enveloped-based Selection 
Algorithm; Knowles et al, 2000). 



2.2 Non-dominated sorting genetic algorithm 

The Non-dominated Sorting Genetic Algorithm (NSGA) was proposed by Srinivas and Deb 
(1994) and is based on the concept of non-domination in a Pareto sense as explained 
previously. 

The basic difference of NSGA in relation to a standard GA is the way in which the 
individuals are evaluated and selected. Basically, in order to obtain the fitness value of an 
individual, instead of using the fitness components associated with each objective involved 
in the problem, these components are used to rank the individuals according to their degree 
of domination over the others in the population; this measure of domination is used to 
define the fitness value that guides the action of the genetic operators and the selection 
process. 
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In order to work out that ranking, the population is organised in several layers of non- 
domination, the outermost layer representing the non-dominated individuals, and the 
innermost containing the most dominated ones. Initially, all individuals in the population 
that are non-dominated are separated as the first, outermost layer, and a (dummy) fitness 
value is assigned to them, whose role is simply to characterise their degree of domination 
over the others in the population. The highest fitness value is assigned to the outermost 
layer, as the individuals in it exhibit the highest degree of domination (the actual value 
assigned is proportional to the population size; this and other details are being omitted here 
for brevity). Then, the remaining individuals are classified again, also based on the non- 
domination criterion, and the second layer is formed with a (dummy) fitness lower than the 
first one. This process continues until all individuals are classified in their respective layers. 
Once the non-domination layers were obtained, in order to maintain the diversity of the 
individuals inside each layer, a sharing function method is used to assign fitness to each 
individual (Goldberg, 1989). The basic idea of the sharing function is to reduce the fitness of 
the individuals that have many neighbours in the same layer. The fitness reduction is by a 
factor that depends on the number and proximity of neighbouring individuals, so that, the 
larger the number of individuals lying close to each other, the higher their fitness reduction; 
analogously, the more isolated an individual, the smallest the influence of the fitness 
reduction. As a consequence, the sharing function induces the formation of niches, thus 
increasing diversity in the population. The sharing function value between two individuals 
in the same border is given by: 

fi _ _ v I r j 
' J CI ij CJ share 

where, dy is the distance between the phenotypes of individuals i and j of the same front; 
Ushare is the maximum distance allowed to two individuals being considered members of the 
same niche; and a is the power that defines as the sharing degree decreases with the 
proximity. 

A niche counter parameter is calculated through the addition of the value of the sharing 
function mentioned above for all individuals of the current front. The shared fitness values 
of each individual are calculated by the division of its dummy fitness by its niche counter. 
After the non-domination classification and fitness assignment are performed, a 
proportional selection scheme is used so that the outer the layer an individual is in, the 
likelier its chance to reproduce. Stochastic remainder selection was the actual scheme used, 
preserved here because it was present in the original proposition of the NSGA. In this 
method, the next generation is formed by taking as many copies of every individual in the 
current population as the result of the integer division of its fitness value by the average 
fitness of the population. Subsequently, if the population needs be filled in, the fractional 
part of the latter division is used (Goldberg, 1989). The other steps of the NSGA are very 
similar to the simple GA (Goldberg, 1989). 
Figure 2 shows a schematic flow chart of the NSGA. 

Therefore, no elitist strategy is used in NSGA, in the sense of an external archive that would 
retain the best non-dominated solutions found along the evolutionary process, that would 
directly pass on to the next generation, regardless of how well fitted those individuals might 
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already be. This is in contrast with various more recent, elitist, multi-objective evolutionary 
methods (Coello et al, 2002), such as NSGA-II (Deb et al, 2000), SPEA (Zitzler & Thiele, 
1999), SPEA2 (Zitzler et al, 2001), PAES (Knowles & Cornes, 1999) and PESA (Knowles et al, 
2000). 
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Fig. 2. NSGA's flow chart 



3. Data mining 

3.1 Knowledge discovery in databases 

Data mining (DM) is the process of identification and extraction of useful information in 
typically large databases. DM aims to automatically discover the knowledge that is not 
easily perceivable. Data mining is considered the central step of the high-level process called 
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Knowledge Discovery in Databases (KDD). KDD includes includes the application of 

several preprocessing methods aimed at facilitating the application of the data mining 

algorithm and postprocessing methods aimed at refining and improving the discovered 

knowledge. 

Data preprocessing aims to assure the data quality and to reduce the problem complexity, 

consolidating the relevant information. It commonly includes the following steps (Freitas, 

2002): 

i. data integration: involves, for instance, removing inconsistencies in attribute names or 

attribute value names between data sets of different sources. 
ii. data cleaning: may involve detecting and correcting errors in the data, filling in missing 

values, etc. 
in. discretization: consists of transforming a continuous attribute into a categorical (or 

nominal) attribute, taking on only a few discrete values. 
iv. attribute selection: consists of selecting a subset of attributes relevant for classification, 

among all original attributes. 
Data postprocessing goal is to improve the understanding of the knowledge discovered, 
validating it through quality measures of the solution and the data analyst perception. It can 
be applied, for example, to simplify the discovered rule set by removing some rules and/ or 
rule conditions, in order to improve knowledge comprehensibility for the user. Another goal 
of postprocessing can be to extract a subset of interesting rules, among all discovered ones. 
Methods for selection of interesting rules can be divided into subjective methods, user- 
driven and domain-dependent; and objective ones. 

3.2 Data mining tasks 

There are different types of tasks associated to data mining process. Each task can be 
thought of as a particular kind of problem to be solved by a data mining algorithm.The main 
types of tasks performed by DM algorithms are: 
i. classification: prediction of the value (the class) of a user-specified goal attribute based 

on the values of other attributes, called the predicting attributes. 
ii. association: discovery of association rules typically extracted from databases in which 

each record consists of a set of binary attributes called items. An association rule is a 

relationship of the form IF X THEN Y, where X and Y are sets of items and X n Y = 0. 
Hi. clustering: creation of groups (clusters) determined by the data set, based on similarity 

measures or probabilistic models. 
iv. Regression: association of an item to one or more prediction variables of real values. 
v. sequences analysis: determination of the sequential characteristics, for example, in data 

with time dependence, which idea is the extraction and register of deviations and 

trends in the time. 
vi. summarizing: description in compact form of a data subgroup. 
vii. dependence modeling: prediction of the values of several user-specified goal attributes 

based on the values of other predicting attributes, being that a goal atribute X can be a 

predictive atribute of another goal atribute Y. 
Dependence modelling is the task investigated in this work. The most popular DM tasks are 
classification and association. Classification is the most investigated one. It searches for the 
knowledge able to predict the value of a previously defined goal attribute based on other 
attributes (prediction attributes). This knowledge is often represented by IF-THEN rules. 
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Dependence modeling task can be seen as a generalization of the classification task. 
Dependence modeling task also aims to discover rules able to predict the goal attribute 
value, from values of the prediction attributes. However, in dependence modeling, there are 
more than one goal attribute. Initially, a small set of goal attributes whose prediction is 
considered interesting is specified. These attributes may occur in the consequent part of the 
rule (THEN) or in the antecedent part of the rule (IF). The others attributes occur only in the 
rule antecedent. As a consequence, dependence modeling task has a larger search space than 
the classification task, when one considers the same data set for both tasks. 
Dependence modeling task is also different from the discovery of association rules. In 
association task, any attribute of the analyzed database can occur in the antecedent or in the 
consequent part of a rule. By other hand, only few pre-selected attributes - the goal 
attributes - can occur in the consequent part when the dependence modeling is considered. 
Also, in dependence modeling task, these same goal attributes can occurs in the antecedent 
part of a rule. 

3.3 Evolutionary algorithms applied to data mining 

Depending on the goal of the related mining process, a different class of DM algorithm is 
more appropriate. Some of the main techniques used in data mining are neural networks 
and decision trees. Evolutionary methods, especially the genetic algorithms (GA) (Goldberg, 
1989), come also being investigated in several DM problems. Accordinly to Freitas (2002), 
the main motivation for using GAs in the discovery of high-level prediction rules is that 
they perform a global search and cope better with attribute interaction than the greedy rule 
induction algorithms often used in data mining. 

The most relevant aspects in GA specification are the individual's encoding and the fitness 
function, which must represent the mined information and the quality of this information, 
respectively. 

Two approaches have been used to represent IF-THEN rules as GA individuals: Michigan 
and Pittsburgh (Freitas, 2002). In the Michigan approach, each individual represents a single 
rule and the population as a whole represents a set of rules. In Pittsburgh approach, each 
individual represents a full set of rules. The kind of rule which is wanted to discover and the 
kind of data mining task addressed in the application can determine the choice between 
these two approaches. For example, in a classification task it is usually to evaluate the 
quality of the rule set as a whole, rather than the quality of a single rule. Therefore, the 
interaction among the rules is important indicating the Pittsburgh approach as more 
appropriated. The Michigan approach might be more adequate in other kinds of data 
mining problems as in a task where the goal is to find a small set of high-quality prediction 
rules, and each rule is often evaluated independently of other rules. Another aspect 
important to this choice is that using the Michigan approach, GA are more suitable for a 
good convergence as the individual encoding is simpler than in Pittsburgh's. In this work, 
the Michigan approach was adopted. 

The evaluation of each individual is based on the metrics used to measure the quality of the 
rule codified. Several metrics can be used but usually only those related to the prediction 
accuracy of the rules are used, such as confidence factor, coverage, sensitivity and 
specification. For example, in (Fidelis et ah, 2000) the quality of the individual (rule) is 
measured by multiplying specificity and sensitivity. Freitas (2002) has pointed out the 
importance to include other metrics in the rules evaluation, such as, comprehensibility and 
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interestingness (Carvalho et al, 2003). This raises the question of how to evaluate the trade- 
off between the accuracy of a model and other quality criteria. A typical example is a 
scenario where one wants to maximize both the accuracy and the simplicity of a 
classification model. This scenario configurates the data mining task as a multiobjective 
problem. The "conventional" approach to deal with this scenario is transforming the multi- 
objective problem into a single-objective one using a weighted formula. For example, a 
standard genetic algorithm was used by Noda and collaborators (1999) to search for rules 
evaluated by two metrics, one related to interestingness and another related to accuracy. 
Genetic algorithm aggregates theses two metrics into a single fitness, through of a pondered 
average. 

Freitas (2004) presented a critical analysis about the three main multi-objective approaches 
to cope with data mining tasks: 
i. Transforming the original multi-objective problem into a single-objective one by using a 

weighted formula, as the single-objective GA previouly cited and described in (Noda et 

al, 1999). 
ii. Manipulating simultaneous and independent objectives with the use of Pareto-based 

GA, as the approach investigated in this work and described in the next section. 
Hi. The lexicographical approach, where the objectives are ranked in order of priority, as 

the work described in (Goncalves et al, 2005) and (Pila et al, 2006). 
Although the weighted formula approach is the most popular approach in the data mining 
literature, Freitas pointed out some drawbacks related to this approach. One of these 
drawbacks is that it mixes different non-commensurable model-quality criteria into the 
same formula. In particular, this has the disadvantage of producing model-quality measures 
that are not very meaningful to the user, going against the principle that in data mining 
discovered knowledge should be not only accurate but also comprehensible to the user. This 
and other drawbacks of the weighted-formula approach are avoided by the Pareto 
approach. 

Pareto-based multi-objective GAs have been previously investigated in DM tasks, as the 
algorithms used in the search for nonlinear models of direct marketing in (Bhattacharyya, 
2000) and to select characteristics for non-supervised learning in (Kim et al, 2000). In (Iglesia 
et al., 2003), an approach based on the NSGAII is employed in the classification task (only 
one goal attribute), using metrics related to the rules precision: accuracy and covering. A 
Pareto-based GA was implemented to mine rules from market-basket type databases 
(association task), using metrics for accuracy, comprehensibility and degree of 
interestingness (Ghosh & Nath, 2004). An approach based on the NSGAII was also applied 
to the association task, for the simultaneous maximization of covering and confidence 
metrics (Ishibuchi et al., 2006a). Other related works that investigate multi-objective data 
mining are references (Francisci & Collard, 2003; Pappa et al., 2004; Lane & Gobet, 2005; 
Ishibuchi et al, 2006b; Berlanga et al, 2006). 

4. Mo-miner, a multi-objective data mining environment 

A data mining environment was implemented to obtain interesting and accurate rules. It 
was named MO-miner (Multi-Objective-based miner). It employs a multi-objective GA 
inspired by NSGA and NSGAII methods that manipulates the metrics accuracy and degree 
of interestingness of the rules, which are kept as independent objectives. This environment 
is the major investigation of this work and it was implemented based on the model 
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described in (Noda et al, 1999), called GA-nuggets. The major modification refers to the 
multi-objective approach, because GA-nuggets uses a single-objective GA and evaluates the 
metrics accuracy and degree of interestingness through a pondered average. 
The individual's encoding represents one candidate rule (Michigan approach). The set of 
goal attributes must be predefined. The antecedent of the rule is formed by a conjunction of 
conditions of the form A\ = Vy, where A\ is the i-th attribute and Vy is the j-th value of this 
attribute's domain. The consequent consists on a simple condition of the form Gk = Vu, 
where Gk is the k-th goal attribute and Vu is the l-th value of its domain. Each individual 
represents the antecedent of a rule. The individual is encoded by a fixed size string, with n 
genes that represent the values of each one of the attributes, as illustrated in Figure 3. This 
encoding must represent a variable length rule. If an attribute does not occur in the rule 
antecedent, the value of its gene is "-1". For each individual, the GA automatically chooses 
the best goal attribute to put in the consequent (Gk = Vu), that is, the values k and I in order 
to maximize rule's evaluation. 



Ai 



A 2 A 3 



A n 



Vi 


v 2 


v 3 




v n 



Fig. 3. Individual representation 

The fitness function must reflect the goal of mining accurate and interesting rules. To do 

this, two metrics were used: Antlnt and PredAcc. Antlnt is the degree of interestingness of 

the antecedent part and PredAcc is the predictive accuracy of the rule. They are defined 

following. 

The rule's predictive accuracy (PredAcc) is an adaptation of the sensitivity indicator, usually 

employed to measure the predictive ability of any rule: 



PredAcc = 



(\A&C\- 0.5) 



(1) 



where | A&cC | is the number of registers in the database that satisfy both antecedent and 
consequent of the rule (true positives); and | A | is the number of cases that satisfy only the 
rule antecedent (true positives plus false positives). The subtraction of 0.5 in the numerator 
was included to penalize rules covering few training examples. This factor is not used when 
the PredAcc is calculated in the test set, because it contains data unseen during training. 
The degree of interestingness of the rule antecedent (Antlnt) is calculated by an information- 
theoretical measure (Freitas, 1998) given by: 



Antlnt - 1 - 



X InfoGain(A t ) In 



(2) 



log 2 \dom(G k 

where n is the number of attributes in the antecedent; dom(Gk) is the number of possible 
values of the attribute Gk occurring in the consequent and InfoGain(Aj) is the information 
gain associated to the attribute A\, given by: 
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InfoGain(A) = Info(G k ) - InfofG^A) (3) 

Info{G k ) = -2(Pr(r„)log 2 (Pr(r w ))) (4) 

i=\ 

"i m k 

Info(G k \Ai ) =1 (MVy X-Z (Pr(^. ) log 2 (Pr(^. )))) (5 ) 

7=1 /=1 

where m^ is the number of possible values of the attribute Gu; m is the number of possible 

values of the prediction attribute Af, Pr(X) denotes the probability of X and Pr(X | Y) denotes 

the conditional probability of X given Y. 

The fitness calculation starts with two objectives, Antlnt and PredAcc, which are kept 

separated, instead of working with an aggregation of these metrics. Subsequently, these 

objectives are used to perform the classification of non-dominated fronts, in the Pareto 

sense, in a similar way to that performed in NSGA. After the initial population of rules was 

generated, each individual is evaluated with respect to the two metrics by equations (1) and 

(2). These values are used to classify the individuals in hierarchical non-dominated fronts. 

Dummy fitness value is attributed to the individuals from each front. Later, these values are 

adjusted for each rule in a same front through the calculation of the sharing function and by 

the niche counter (Srinivas & Deb, 1994). After this, each individual finally receives a scalar 

fitness and the children generation is processed by four steps: 

i. Selection of crossover pairs, 

ii. Uniform crossover with eventual repair of invalids chromosomes, 

Hi. Random mutation, 

iv. Insertion and removal of genes. 

These steps are detailed as follows. 

A simple tournament selection is used, where a group (Tour) of individuals is randomly 

sorted and the one with the highest fitness is chosen for reproduction. It is important to note 

that a stochastic remainder selection was used in the original proposition of the NSGA. We 

adopted Tour = 3 in the tournament selection, in which each pair of individuals are selected 

from two groups of 3. The criterion used to define the tournament winner is: a pair <goal 

attribute, value> is drafted, the rules participants of the tournament are compared with 

respect to this pair and the one with the highest fitness is chosen. 

After the selection, a uniform crossover operator was used with a probability of 70% for 

applying crossover to a pair of individuals and another probability of 50% for swapping 

each attribute's value in the rule antecedents of two individuals. After crossover is done, GA 

verifies if any invalid individual was created. If so, it uses a repair operator to produce valid 

individuals. 

The mutation operator randomly transforms the value of an attribute into another value 

belonging to the domain of that attribute. The mutation rate used was 5%. 

Besides crossover and mutation, there are the insertion and removal operators. These 

operators randomly insert or remove a condition in the antecedent. Their probabilities 

depend on the number of attributes in the antecedent. The insertion operator has a null 

probability of application when the rule antecedent has the maximum number of attributes 

(as specified by the user). The removal operator works in opposite way. The used insertion 

and removal rates are 50% and 70%, respectively. 
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The GA uses a kind of elitism (Goldberg, 1989) in the population reinsertion after each 
generation. Elitism strategy keeps the M best prediction rules for each pair <goal attribute, 
value> to the next generation, among K possible pairs, corresponding to an elitism factor of 
K*M. In the experiments described in section 6, the number of rules kept by elitism adopted 
was 3 rules for each pair. 

It is important to note that the original NSGA is a non-elitist method. The fact that we 
centred the multi-objective approach herein around a non-elitist method, instead of using its 
more recent, elitist counterpart, NSGA-II (Deb et ah, 2000), derives from the problem being 
tackled. The point is that, the single-objective genetic algorithm presented in (Noda et al., 
1999) (GA-nuggets) - that has been used as the basis of our multi-objective GA - relies on a 
elitist strategy for reinsertion n of individuals in the population; hence, to maintain all the 
basic structure of that reference GA was a natural constraint, at the same time that using a 
non-elitist multi-objective algorithm seemed a sensible decision, so as not to overload the 
resulting algorithm as far as elitism is concerned. But now that the simpler, and to some 
extent naive multi-objective approach we adopted has shown to be successful, changing the 
actual multi-objective algorithm to a more sophisticated and well-established one, is clearly 
a natural step ahead. 

A population size of 100 individuals and a number of 50 generations were used. Although 
MO-miner goal is to find a front with several non-dominated solutions, a criterion to select 
the best rule, in end of GA execution, must be established. We applied the same function 
adopted in (Noda et al, 1999) as the single-objective fitness function and it consists of two 
parts. The first one measures the degree of interestingness of the rule as a whole (antecedent 
and consequent), while the second measures its predictive accuracy. Equation (6) was 
applied to decide which is the best rule in the highest non-dominated front: 

_. Antlnt + Conslnt . . . .. 

Fitness - w x x hw 2 x PredAcc (6) 

where Antlnt is the degree of interestingness of the antecedent, PredAcc is the predictive 
accuracy of the rules; and wi and w>2 are user-defined weights and we used the same values 
adopted by Noda and collaborators (1999), wi = 1/3 and wi = 2/3, that gives a greater 
weight to the rule accuracy. 

Conslnt is the degree of interestingness of the consequent part (Noda et al, 1999) and it is 
calculated by: 

Conslnt = ^l-Pr^) (7) 

where Pr(Vu) is the relative frequency of the goal attribute value Vki. 

Accordingly to Freitas (2004), one of the arguments against Pareto approaches to data mining 
algorithms is the difficulty of choosing a single "best" solution to be used in practice. A possible 
criticism of such kind of approach is that the data mining algorithm returns a set of non- 
dominated solutions, whereas in practice the user will often use a single solution. Therefore, a 
difficult problem associated with the Pareto approach is to decide how can one choose the 
"best" non-dominated solution, out of all non-dominated solutions in the final generation. 
Choosing the best solution based on Equation (6), we define GA's final output in an objective 
way and we avoid the problem cited above. Besides, the rules are selected with the same final 
criterion used in the single-objective environment implemented in (Noda et al., 1999) what turns 
the results obtained by the two approaches suitable for a comparative analysis. 
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The metric Conslnt, given by equation (7), was not employed as an independent objective of 
our multi-objective approach; because it returns a constant value that depends only on the 
consequent pair <goal attribute, value> at issue and on the number of occurrences of this pair 
in the database. However, it is used in the final evaluation of the best individual to weight 
the degree of interestingness of the rule in the same way as done in (Noda et al, 1999). 
Figure 4 presents a schematic flow chart of MO-miner environment. 
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Fig. 4. MO-miner' s flow chart 
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5. Data sets 

We used two data sets to evaluate MO-miner: Nursery and Zoo. These data sets were also 
explored in (Noda et ah, 1999). Both databases are public data sets and they were obtained 
from the UCI repository (www.ics.uci.edu/~mlearn/MLRepository.html) of machine 
learning databases. 

Nursery database was obtained in http://archive.ics.uci.edu/ml/datasets/Nursery, 
available in this repository since 1997. It was derived from a hierarchical decision model 
originally developed to rank applications for nursery schools. It was used during several 
years in 1980's when there was excessive enrollment to these schools in Slovenia, and the 
rejected applications frequently needed an objective explanation. The final decision 
depended on three subproblems: occupation of parents and child's nursery, family structure 
and financial standing, and social and health picture of the family. The database contains 
12960 registers and nine categorical attributes: parents, health, form, children, has_nurs, 
housing, finance, social and recommendation. Three attributes was used as goal: finance, 
social and recommendation. Finance has two possible values (convenient and inconv), social 
has three possible values (non-prob, slightly-prob and problematic) and recommendation 
has five values in its domain (not-recom, recommended, very-recom, priority, spec-prior). It 
results in rules with ten different consequents: 

Consequent 1: finance = convenient 

Consequent 2: finance = inconv 

Consequent 3: social=non-prob 

Consequent 4: social = slightly-prob 

Consequent 5: social =problematic 

Consequent 6: recommendation = not-recom 

Consequent 7: recommendation = recommended 

Consequent 8: re-commendation = very-recom 

Consequent 9: recommendation = priority 

Consequent 10: recommendation = spec-prior 
Zoo is the second public database used and it was also obtained from the UCI repository 
(http://archive.ics.uci.edu/ml/datasets/Zoo), and it is available in this repository since 
1990. Each instance of this database corresponds to an animal. It contains only 101 registers 
and a total of eighteen categorical attributes: hair, feathers, eggs, milk, backbone, fins, legs, 
tail, catsize, airbone, aquatic, breathes, venomous, toothed, predator, domestic and type. 
Three goal attributes was chosen: predator, domestic and type. Predator and domestic are 
binary attributes and type has seven possible numerical values (from 1 to 7). Therefore, 
there are eleven different consequents: 

Consequent 1: predator=0 

Consequent 2: predator=l 

Consequent 3: domestic=0 

Consequent 4: domestic=l 

Consequent 5: type=l 

Consequent 6: type=2 

Consequent 7: type=3 

Consequent 8: type=4 

Consequent 9: type=5 
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Consequent 10: type=6 
Consequent 11: type=7 



6. Experiments 

6.1 Single-objective environment 

Besides MO-miner detailed in Section 4, a second environment was implemented to supply 
comparative results to evaluate the effectiveness of the multi-objective approach. It was 
implemented strongly based on the model described in (Noda et ah, 1999), called GA- 
nuggets. It uses a single-objective standard genetic algorrithm with the same goal of MO- 
miner: to discover accurate and interesting rules. 

Each individual is evaluated through the pondered average described by Equation (6). The 
components TredAcc, Antlnt and Conslnt of Equation (6) was calculated in the same way as 
described to MO-miner and they are given by equations (1), (2) and (7), respectively. The 
user-defined weights w\ and u>2 in Equation (6) was also adopted as in (Noda et ah, 1999): 
wi = 1/3 and Wi = 2/3. 

The single-objective GA also uses a population size of 100 individuals and a number of 50 
generations per execution. After the initial population of rules was generated, each 
individual is evaluated with respect to Equation (6). The children generation is processed as 
in the multi-objective environment: 
i. Simple tournament selection to crossover (Tour = 3). 
ii. Uniform crossover with eventual repair of invalids chromosomes. 
Hi. Random mutation. 
iv. Insertion and removal of the genes. 

Some small modifications had been made in the single-objective GA with respect to the one 
described in (Noda et al, 1999): 

i. We adopted Tour = 3 in the tournament selection. 
ii. The criterion used to define the tournament winner is: a pair <goal attribute, value> is 

drafted, the rules participants of the tournament are compared with respect to this pair 

and the one with the highest fitness is chosen. 
Hi. The number of rules kept by elitism is 3 rules for each pair. 

6.2 Experiments description 

Two kinds of experiments had been executed with the single-objective GA and with MO- 
miner. The first one used the full data set during GA evolution, getting the better rules to 
predict each one of the possible consequents Gk = Vu. In the second experiment, a cross- 
validation was used to evaluate the quality of the discovered rules. Hence, the data set is 
divided into five mutually exclusive and exhaustive partitions. This second kind of experiment 
is composed by five types of GA evolution, each one using a different partition as test set. All 
partitions, except the test set, had been used during the rules evolution as training set, 
corresponding to the data effectively used by the genetic algorithm during rule search. Later, 
the rules are evaluated concerning their prediction accuracy in the test partition. The final 
result is defined by the average results considering the five test partitions. 

6.3 Results using Nursery database 

The results of this section was previously discussed in (Oliveira et ah, 2008). 
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6.3.1 Full set experiment 

Both single-objective GA and MO-miner had been applied to 100% of the registers from 
Nursery database. In each experiment, 10 different GA executions had been performed. 
Table 1 presents Antlnt and PredAcc values for the best rule obtained in these executions, for 
each consequent. Figure 5 shows radial charts of these metrics to aid the comparative 
analysis: (a) Antlnt and (b) PredAcc. In general, both environments had mined rules with a 
high degree of interestingness and a good accuracy for the majority of the consequents. The 
single-objective GA found rules with degree of interestingness below of 0.95 in consequents 
6, 8, 9 and 10 and MO-miner only in consequents 8 and 10. In all the others consequents, 
Antlnt exceeds 0.97. Both the single-objective and MO-miner had found bad rules (< 0.5) for 
consequents 3, 4 and 7 with respect to accuracy. These results are compatible with those 
reported in (Noda et al, 1999). 

In a comparative analysis, one can observe that the most significant change occurs with 
respect to the Antlnt, while the metric PredAcc remains practically unchanged for both 
environments. Concerning degree of interestingness, MO-miner returned rules better than 
single-objective GA in seven consequents, with an increase above to 10% in two of them 
(consequents 6 and 9). With respect to accuracy, the multi-objective returns rules a little bit 
higher than the single-objective environment in three consequents (consequents 6, 9 and 10) 
and equivalent in the others. 

We conclude that our multi-objective approach found rules with higher degree of 
interestingness, without damage on the prediction accuracy. 

Table 2 presents the best rules discovered by MO-miner. Their quality of prediction is better 
estimated by the average of PredAcc obtained in cross-validation experiments, described as 
follows. 



Consequent 


Single-Objective 


MO-miner 


Antlnt 


PredAcc 


Antlnt 


PredAcc 


1 


0.999 


0.987 


0.999 


0.987 


2 


0.999 


0.992 


0.999 


0.992 


3 


0.993 


0.498 


0.999 


0.498 


* 


0.993 


0.498 


0.999 


0.498 


5 


0.996 


0.994 


0.999 


0.994 


6 


0.891 


0.998 


0.993 


0.999 


7 


0.978 


0.500 


0.998 


0.500 


3 


0.921 


0.969 


0.921 


0.969 


9 


0.907 


0.996 


0.998 


0.998 


10 


0.908 


0.997 


0.909 


0.999 



Table 1. Nursery Database: Antlnt and PredAcc obtained with single-objective GA and MO- 
miner in full set experiment. 



6.3.2 Cross-validation experiment 

The cross-validation method was used to verify the generalization capability of the mined 
rules. Nursery database was divided into five partitions and five different GA evolutions 
had been performed. In each kind of evolution, a different partition was used as the test set 
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and the others four partitions are merged and they were used in the evolutionary search for 
the rules (training). Ten different GA executions had been performed for each test partition. 
After the best rules for each consequent were obtained, they had been applied to the test 
partition to evaluate its prediction over examples not seem during the evolution. 
Table 3 presents PredAcc average for the best rules found for each environment, both in 
training and in test. Figure 6 shows radial charts of the metric PredAcc (a) in training and (b) 
in test. 
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Fig. 5. Single-objective GA and MO-miner results in the full Nursery database: (a) Antlnt (b) 
PredAcc. 
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IF (health = recommended) AND (form = foster) AND (housing = convenient) AND 

(recommendation = very_recom) 

THEN (finance= convenient) 



IF (frousztt£= convenient) AND (social = HOHprofr) AND 

(health = recommended) AND (recommendation = spec_prior) 

THEN (finance= inconv) 



IF (health = recommended') AND (recommendation = very_recom) 
THEN (social = non_prob) 



IF (health = recommended'} AND (recommendation = very_recom) 
THEN (social = sli%htly_prob) 



IF (fro«sztt£= convenient) AND (finance = convenient) AND (healih= recommended) AND 

(recommendation = spec_prior) 
THEN (social = proUematic) 



IF (has_nurs = proper) AND (finance = convenient) AND (social = nonprob) AND 

(health = not_recom) 
THEN (recommendation = not_recom) 



IF (par en ts= usual) AND (has_nurs=proper) AND (finance = coHE^ttieHfc) AND 

((Mtfre^l) AND (/orm = compter) AND (housing = convenient) AND 

(social Honprofr) AND (health = recommended) 

THEN (recommendation = recommended) 



IF (parents = usual) AND (has_nurs= less_proper) AND (housing = ooH&enieHt) AND 

(/znflnc£ = con E^ttient) AND (social = slightly _prob) AND (hariih = reoommeH^tf) 

THEN (rtfoofflffltfrttiitiott = very_recom) 



IF (htis_Hure = proper) AND (housing = convenient) AND 

(finance= convenient) AND (fteoiiJT; = priority) 

THEN (recommendation = priority) 



IF (hfls_«urs = very_crit) AND (finance = inconvenient) AND (socaai = 

(health = recommended) 
THEN (recommendation = spec_prior) 



problematic) AND 



Table 2. Rules extracted by MO-miner from Nursery database. 



Consequent 


Single -Objective 


MO-miner 


Training 


Test 


Training 


Test 


1 


0.934 


0.400 


0,985 


0,600 


2 


0.992 


0.700 


0,991 


0,700 


3 


0.499 


0.263 


0,499 


0,333 


4 


0.493 


0.200 


0,493 


0,367 


5 


0.994 


0.636 


0,993 


0,733 


6 


0.997 


0.300 


0,999 


1,000 


7 


0.500 


0.007 


0,500 


0,007 


3 


0.963 


0.000 


0,972 


0.000 


9 


0.993 


0.644 


0,999 


0,450 


10 


0.995 


0.590 


0,993 


0,590 



Table 3. Nursery Database: PredAcc of training and test obtained with single-objective GA and 
MO-miner in cross-validation experiment. 
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In both environments, the average of PredAcc in the training set is close to the values 
obtained with the full set. However, the average of PredAcc in the test set falls substantially, 
indicating that the rules mined in training do not exhibits good generalization ability. When 
comparing results of the two environments one can notice that even so the performance has 
been very similar in the training sets, MO -miner returned higher PredAcc rules in the test 
sets. PredAcc average measured in test sets for MO-miner had been higher in five 
consequents (1, 3, 4, 5 and 6) and lower in only one (consequent 9). Thus, it is possible to 
conclude that the rules mined by MO-miner present better generalization ability than those 
obtained by simple-objective GA. 
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Fig. 6. PredAcc average values of the single-objective GA and MO-miner environments in 
cross-validation experiment using Nursery database: (a) training (b) test. 
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6.4 Results using Zoo database 
6.4.1 Full set experiment 

Table 4 shows the values of Antlnt and PredAcc for the best rules found using the full Zoo 
database. Both environments (single-objective and MO-miner) were able to mine rules with 
high degree of interestingness and a good accuracy for almost all the consequents; the 
unique exception was the consequent 4. The single-objective GA found rules with Antlnt 
below of 0.9 in consequents 5 to 11 while MO-miner only in consequents 7, 8 and 9 (but in all 
these case Antlnt was at least 0.84). In the other cases, Antlnt is above 0.95 for both 
environments. In respect to accuracy, single-objective and MO-miner had found bad rules 
(< 0.8) only for consequent 4. Again, these results are compatible with (Noda et ah, 1999). As 
observed in Nursery database, the most significant difference occurs with respect to Antlnt, 
while PredAcc remains practically equivalent for both environments. 

Figure 7 shows the radial charts to aid the comparative analysis: (a) Antlnt and (b) PredAcc. 
Concerning Antlnt, MO-miner returned rules better than single-objective GA in seven 
consequents, with an increase above to 10% in four of them (5, 6, 10 and 11). With respect to 
PredAcc, MO-miner returns rules a little bit higher in four consequents (3, 5, 6 and 8), a lit bit 
lower in one (consequent 7), and equivalent in the others. As observed in Nursery 
experiments, the major conclusion is that the multi-objective approach found rules with 
higher degree of interestingness, without damage on the prediction accuracy. 
Table 5 presents the best rules discovered by MO-miner using the full Zoo database. Cross- 
validation experiments described in the next section can better estimate their quality of 
prediction. 



Consequent 


Single-Objective 


MO-miner 


Antlnt 


PredAcc 


Antlnt 


PredAcc 


1 


0.956 


0.929 


0.999 


0.929 


2 


0.933 


0.958 


0.999 


0.953 


3 


0.990 


0.930 


0.950 


0.936 


4 


0.996 


0.500 


0.996 


0.500 


5 


0.330 


0.934 


0.932 


0.933 


6 


0.393 


0.971 


0.932 


0.975 


7 


0.349 


0.375 


0.331 


0.333 


3 


0.337 


0.953 


0.340 


0.962 


9 


0.333 


0.375 


0.363 


0.375 


10 


0.343 


0.937 


0.932 


0.937 


11 


0.337 


0.937 


0.932 


0.937 



Table 4. Zoo Database: Antlnt and PredAcc obtained with single-objective GA and MO-miner 
in full set experiment. 



6.4.2 Cross-validation experiment 

As performed for the Nursery database, the Zoo database was divided into five partitions 
and five different GA evolutions had been performed. For each test partition, ten different 
GA executions had been performed. The best rules obtained for each consequent had been 
applied to the test partition to evaluate its prediction. 
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Table 6 presents PredAcc average for the best rules found (over the 5 partitions), both in 
training and in test. Figure 8 shows radial charts of the metric PredAcc (a) in training and (b) 
in test. The average of PredAcc in the training set is close to the values obtained with the full 
set, as for the single-objective GA as for MO-miner. But as observed in Nursery experiments, 
the average of PredAcc in the test set falls substantially, indicating that the rules mined in 
training do not exhibits good generalization ability, for both environments. Once more MO- 
miner returned higher PredAcc rules in the test sets even so the performance has been very 
similar in the training sets. PredAcc average measured in test sets for MO-miner had been 
higher in six consequents being this performance significatively in three of them. It is 
possible to conclude that, as concluded in Nursery experiments, the rules mined by MO- 
miner present better generalization ability. 
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Fig. 7. Single-objective GA and MO-miner results in the full Zoo database: (a) Antlnt 
(b) PredAcc. 
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IF (domestic! ) AND (ai£szze=0) 
THEN (predatQ?=Q) 



IF (airhone^O) AND (aquatic=l ) AND (at£szze=l ) 
THEN (predtttor=l) 



IF (fctizr=0) AND (predalo?=l ) 
THEN ((fom£stic=0) 



IF (e££S=0) AND (vetiQmoiis=0) AND (tazl=0) AND (ctitsize=0) 
THEN(tfomes£z'c=l) 



IF (mzffipl) AND (vetiomotis=0) 
THEN(^p^l) 



IF {Jmihers=l ) AND (venomous=0) 
THEN (fyp^2) 



IF (hair=0) AND (aquatic=0) AND (pre<fctor=l ) AND (tootfted=l ) AND (dbmesfcze=0) 
THEN (fype=3) 



IF ( £££S=1 ) AND (/m s=l ) AND ( &>m esti^O ) 
THEN (frpe=4) 



IF (/aitfters=0) AND (flz'rfctt£=0) AND (aquatia=l ) AND (&rai^es=l ) AND (m£siz£=0) 
THEN (typ^5) 



IF(aqitatic=0) AND (/zhs=0) AND (lfi£S=6) AND (ojtsize=0) 
THEN(fyp^6) 



IF (flir£oH^=0) AND (pret£itor=l ) AND (fridtfoHtf=0) AND (dome$tic=0) 
THEN(hffM=7) 



Table 5. Rules extracted by MO-miner from Zoo database. 



Consequent 


Single -Objective 


MO-miner 


Training 


Test 


Training 


Test 


1 


0.919 


0.200 


0.922 


0.433 


2 


0.947 


0.900 


0.943 


0.933 


3 


0.979 


0.975 


0.933 


0.973 


4 


0.471 


0.050 


0.500 


0.030 


5 


0.930 


1.000 


0.935 


1.000 


6 


0.964 


1.000 


0.970 


1.000 


7 


0.842 


0.400 


0.342 


0.430 


3 


0.943 


0.300 


0.950 


0.300 


9 


0.333 


0.600 


0.353 


0.600 


10 


0.915 


0.500 


0.922 


0.600 


n 


0.917 


1.000 


0.920 


1.000 



Table 6. Zoo Database: PredAcc of training and test obtained with single-objective GA and 
MO-miner in cross-validation experiment. 
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Fig. 8. PredAcc average values of the single-objective GA and MO-miner environments in 
cross-validation experiment using Zoo database: (a) training (b) test. 



7. Conclusion 

The application of a multi-objective GA-based environment named MO-miner, inspired by 
the family of algorithms NSGA and designed to discover accurate and interesting 
dependence modeling rules, was investigated in this work. Our investigation was 
performed by applying MO-miner to two distinct public data sets named Nursery and Zoo, 
composed by 12960 and 101 registers, respectively. 

The accuracy and the interestingness of the rules mined by MO-miner were compared with 
those found by a single-objective standard genetic algorithm, based in its turn on the 
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environment implemented by Noda and collaborators (1999) named GA-nuggets. The results 
obtained with our single-objective GA, both for full database and for crossvalidation 
experiments, are very similar to those reported in (Noda et ah, 1999), in which the same data 
sets were mined. 

Comparing the performance of the both environments, it was possible to observe that, when 
using the full database, they had found rules very similar with respect to accuracy. 
However, MO-miner returned more interesting rules. We believe that this result is due to the 
kind of rule evaluation performed by MO-miner, which kept metrics independents, and to 
MO-miner' s dynamics that drives the search toward the region of the highest values in both 
metrics. Conversely, in single-objective environment, the balance between the metrics was 
pre-defined, giving a higher weight to the accuracy with respect to the degree of 
interestingness. 

In respect to the cross-validation experiments, it was evidenced that the results found by 
MO-miner were better, resulting in a lesser accuracy decay between training and test. This 
indicates that the rules obtained by MO-miner have greater generalization than those 
extracted using the single-objective environment. 

Despite the fact that Zoo and Nursery data sets are very different among each other, specially 
because the first is a small database and the second is a large one, in the comparative 
experiments applying the two environments, we arrived at similar conclusions. Using both 
data sets, MO-miner found more interesting rules in the full set experiment and rules with 
better generalization ability in the cross-validation experiment. 

Different paths related to multi-objective genetic algorithms and data mining can be 
explored as continuity of this study, such as: inclusion of other metrics as comprehensibility 
(Freitas, 2002); use of others multi-objective techniques (Coello et ah, 2002); use of other 
fitness functions, among others. 
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1. Introduction 

In the last years there has been a surge in interest in a topic called social robotics. As used 
here, social robotics does not relate to groups of robots that try to complete tasks together. 
For a group of robots, communication is simple, they can use whatever complex binary 
protocol to "socialize" with their partners. For us, the adjective social refers to humans. In 
principle, the implications of this are much wider than the case of groups of robots. 
Socializing with humans is definitely much harder, not least because robots and humans do 
not share a common language nor perceive the world (and hence each other) in the same 
way. Many researchers working on this topic use other names like human-robot interaction 
or perceptual user interfaces. However, as pointed out in (Fong et al., 2003) we have to 
distinguish between conventional human-robot interaction (such as that used in 
teleoperation scenarios or in friendly user interfaces) and socially interactive robots. In 
these, the common underlying assumption is that humans prefer to interact with robots in 
the same way that they interact with other people. 

Human-robot interaction crucially depends on the perceptual abilities of the robot. Ideal 
interaction sessions would make use of non-invasive perception techniques, like hands-free 
voice recognition or computer vision. Computer vision is no doubt the most useful 
modality. Its non-invasiveness is the most important advantage. 

In this paper, a number of computer vision techniques for human-robot interaction are 
described. All of them have been used in a prototype social robot called CASIMIRO (Figure 
1), an animal-like head that stands on a table and has the goal of interacting with people, see 
(Deniz, 2006) for details. 

2. Omnidirectional vision 

Most social robots built use two types of cameras: a wide field of view camera (around 70 
deg), and a foveal camera. Recently, interest in omnidirectional vision has increased in 
robotics. Omnidirectional vision allows to capture images that span 360°. Four main 
techniques are being used to achieve this: 

• Cameras with fish-eye lenses, which have a very short focal length (called dioptric 
systems. 



430 



Advances in Robotics, Automation and Control 



Cameras with curved (convex) mirrors mounted in front of a standard lens (called 

catadioptric systems). This is the most common variant. 

Sets of cameras mounted in a ring or sphere configuration. The information flow is time 

consuming. 

An ordinary camera that rotates around an axis and takes a sequence of images that 

span 360°. Mechanical looseness can appear. 




Fig. 1. CASIMIRO. 

The omnidirectional camera shown in Figure 2 is a catadioptric setup that gives the robot a 
180 deg field of view, which is similar to that of humans. The camera is to be placed in front 
of the robot. The device is made up of a low-cost USB webcam, construction parts and a 
curved metallic surface looking upwards, in this case a kitchen ladle. As for the software, 
the first step is to discard part of the image, as we want to watch only the frontal zone, 
covering 180 degrees from side to side. Thus, the input image is masked in order to use only 
the upper half of an ellipse, which is the shape of the mirror as seen from the position of the 
camera. 

A background model is obtained as the mean value of a number of frames taken when no 
person is present in the room. After that, the subtracted input images are thresholded and 
the close operator is applied. From the obtained image, connected components are localized 
and their area is estimated. Also, for each connected component, the Euclidean distance 
from the nearest point of the component to the centre of the ellipse is estimated, as well as 
the angle of the centre of mass of the component with respect to the centre of the ellipse and 
its largest axis. Note that, as we are using an ellipse instead of a circle, the nearness measure 
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obtained (the Euclidean distance) is not constant for a fixed real range to the camera, though 
it works well as an approximation. The robot uses this estimate to keep an appropriate 
interaction distance. The background model M is updated with each input frame: 



M(k + 1) = M(k) + U(k) ■ [I(k) -M(k)] 
, where I is the input frame and U is the updating function: 

U(k) = exp(-b-D(k)) 
D(k) = a -D(k-l)+(l-a) ■ | I(k)-I(k-1) | 



(1) 



(2) 
(3) 



a (between and 1) and b control the adaptation rate. Note that M, U and D are images, the 
x and y variables have been omitted for simplicity. For large values of a and b the model 
adaptation is slow. In that case, new background objects take longer to enter the model. For 
small values of a and b, adaptation is faster, which can make animated objects enter the 
model. 




Fig. 2. Omnidirectional camera. 

The method described up to this point still has a drawback. Inanimate objects should be 
considered background as soon as possible. However, as we are working at a pixel level, if 
we set the a and b parameters too low we run the risk of considering static parts of animate 
objects as background too. This problem can be alleviated by processing the image D. For 
each foreground blob, its values in D are examined. The maximum value is found, and all 
the blob values in D are set to that level. Let the foreground blobs at time step k be 
represented as: 
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B i = {Xi j/ y ij };i = l / .. / NB;j = l / . v Ni (4) 

There are NB blobs, each one with H pixels. Then, after (3) the following is applied: 

mi = max j=i„., N i D(xij , yij , k) ; i = 1, ..,NB (5) 

D(xij , yij , k) = mi ; i = 1, ..,NB ; j = 1, ..,Ni (6) 

With this procedure the blob only enters the background model when all its pixels remain 
static. The blob does not enter the background model if at least one of its pixels has been 
changing. 

3. Face detection 

Omnidirectional vision allows the robot to detect people in the scene, just to make the neck 
turn towards them (or somehow focus its attention). When the neck turns, there is no 
guarantee that omnidirectional vision has detected a person, it can be a coat stand, a 
wheelchair, etc. A face detection module should be used to detect people (and possibly 
facial features). Facial detection commonly uses skin-color as the most important feature. 
Color can be used to detect skin zones, though there is always the problem that some objects 
like furniture appear as skin, producing many false positives. Figure 3 shows how this 
problem affects detection in the ENCARA facial detector (M. Castrillon-Santana et al., 2005), 
which (besides other additional cues) uses normalized red and green color components for 
skin detection. 





Fig. 3. Skin color detection. Note that wooden furniture is a distr actor for facial detection. 
Both the bounding box and the best-fit ellipse are rather inaccurate (left). 

In order to alleviate this problem, stereo information is very useful to discard objects that are 
far from the robot, i.e. in the background. Stereo cameras are nowadays becoming cheaper 
and faster. A depth map is computed from the pair of images taken by a stereo camera 
situated under the nose of the robot. The depth map is efficiently computed with an 
included optimized algorithm and library. The map is thresholded and an AND operation is 
performed between this map and the image that the facial detector uses. Fusion of color and 
depth was also used in (Darrell et al., 1998; Moreno et al., 2001; Grange et al., 2002). The 
results are shown in Figure 4. Note that most of the undesired wood colored zones are 
filtered out. 



Practical Computer Vision Techniques for Interactive Robotic Heads 



433 





Fig. 4. Skin color detection using depth information. 



4. Person recognition 

In (Schulte et al., 1999) three characteristics are suggested as critical to the success of robots 
that must exhibit spontaneous interaction in public settings. One of them is the fact that the 
robot should have the capability to adapt its human interaction parameters based on the 
outcome of past interactions so that it can continue to demonstrate open-ended behaviour. 
CASIMIRO is intended to interact with people. Humans will be the most important "object" 
in its environment. Data associated to humans (gathered throughout the interaction) should 
be stored in memory, so that the robot could take advantage of previous experiences when 
interacting with them. Breazeal (Breazeal, 2002) argues that to establish and maintain 
relationships with people, a sociable robot must be able to identify the people it already 
knows as well as add new people to its growing set of known acquaintances. In turn, this 
capacity will be part of the robot's autobiographical memory. 

In order to make this person memory possible, gathered data should be unambiguously 
associated to the correct person. Facial recognition would be the perfect approach. However, 
the experience of the author with face recognition is somewhat negative: face recognition 
still does not work well in unrestricted scenarios. Recognition rates fall as more time passes 
since the training samples were taken. Illumination, pose and expression variations 
normally reduce recognition rates dramatically. 

Colour histograms of (part of) the person's body could also be used as a recognition 
technique. Colour histograms are simple to calculate and manage and they are relatively 
robust. The price to pay is the limitation that data in memory will make sense for only one 
day (at the most). Colour histograms of a person's body were used for short-term 
identification people in (Maxwell, 2003; Kahn, 1996; Maxwell et al., 1999) and also for people 
tracking (Krumm et al, 2000; Collins & Dennis, 2000). 

CASIMIRO achieves person identity maintenance by using colour histograms in conjunction 
with a simple person tracking algorithm. Tracking is done in ID, for the interesting position 
is the angle of the person with respect to the robot. 

The implemented tracking algorithm is very simple. Each person is represented as a single 
point in two sets of horizontal positions (positions range from to 180) at times t - 1 and t. 
The association of points between the two sets is obtained as that which minimizes the total 
sum of distances between points of the two sets. This minimization involves a factorial 
search, though it is practical for the number of people that will be expected to interact with 
the robot. 
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Ties can appear, for example in the case of crossings, see the example of Figure 5. These ties 
are broken by selecting the association with lowest variance of distances, 1 with A and 2 
with B in the case of the example. This always selects non-crossings. 



t-1 



1 2 

(f 1 1 180° 



A B 

( 0° 1 1 180" 

Fig. 5. Tie in sum of distances. The sum of distances 1 1 - A | + | 2 - B | is equal to 1 1 - B | + 
1 2 - A | . Without further information, we cannot know if the two individuals have crossed 
or not. 

Crossings are detected by considering that, in a crossing, there is always a fusion and a 
separation of person blobs. Person blobs are detected by the omnidirectional vision system 
(see above). Fusions and separations are detected as follows: 

• A blob fusion is detected when the number of blobs in the whole omnidirectional image 
decreases by one at the same time that one of the blobs increases its area significantly. 

• A blob separation is detected when the number of blobs in the image increases by one 
at the same time that a fusioned blob decreases its area significantly. 

The only way to know if a there is a crossing is by maintaining some sort of description of 
the blobs before and after the fusion. Histograms of U and V colour components are 
maintained for each blob. The Y component accounts for luminance and therefore it was not 
used. Whenever a separation is detected, the histograms of the left and right separated blobs 
are compared with those of the left and right blobs that were fusioned previously. 
Intersection (Swain and Ballard, 1991) was used to compare histograms (which must be 
normalized for blob size). This procedure allows to detect if there is a crossing, see Figure 6. 
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Fig. 6. Crossings can be detected by comparing blob histograms at fusion and separation 
events. 

The histogram similarities calculated are shown in Figure 7. A crossing is detected if and 
only if (b + c) > (a + d). Note that in the comparison no threshold is needed, making crossing 
detection relatively robust. 
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(blobs right before fusion) 



(blobs right after separation) 



Fig. 7. Blob similarities calculated. 

In order to achieve person identification, a set of Y-U histograms are stored for each person 
detected. The zone from which these histograms are calculated is a rectangle in the lower 
part of the image taken from the stereo camera placed under the nose of the robot. The 
rectangle is horizontally aligned with the centre of the face rectangle detected, and extends 
to the lower limit of the image (chest and abdomen of standing people will always occupy 
that lower part of the image), see Figure 8. The upper edge of the rectangle is always under 
the lower edge of the face rectangle detected. The width of the rectangle is proportional to 
the width of the face rectangle detected. 




Fig. 8. Region used for person identification. 

When the robot fixates on a person that the tracking system has labelled as new (the 
tracking system detects a new person in the scene when the number of foreground blobs 
increases and no blob separation is detected), it compares the histograms of the fixated 
individual with those of previously met individuals. This search either gives the identity of 
a previously seen individual or states that a new individual is in the scene. In any case the 
set of stored histograms for the individual is created/ updated. 



5. Head Nod/Shake detection 

Due to the fact that practical (hands-free) voice recognition is very difficult to achieve for a 
robot, we decided to turn our attention to simpler (though useful) input techniques such as 
head gestures. Head nods and shakes are very simple in the sense that they only provide 
yes/ no, understanding/ disbelief, approval/ disapproval meanings. However, their 
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importance must not be underestimated because of the following reasons: the meaning of 
head nods and shakes is almost universal, they can be detected in a relatively simple and 
robust way and they can be used as the minimum feedback for learning new capabilities. 
The system for nod/ shake detection described in (Kapoor & Picard, 2001) achieves a 
recognition accuracy of 78.46%, in real-time. However, the system uses complex hardware 
and software. An infrared sensitive camera synchronized with infrared LEDs is used to 
track pupils, and a HMM based pattern analyzer is used to the detect nods and shakes. The 
system had problems with people wearing glasses, and could have problems with earrings 
too. The same pupil-detection technique was used in (Davis and Vaks, 2001). That work 
emphasized the importance of the timing and periodicity of head nods and shakes. 
However, in our view that information is not robust enough to be used. In natural 
humanhuman interaction, head nods and shakes are sometimes very subtle. We have no 
problem in recognizing them because the question has been clear, and only the YES/ NO 
answers are possible. In many cases, there is no periodicity at all, only a slight head motion. 
Of course, the motion could be simply a Took up'/'Look down'/ 'Look left'/ 'Look right', 
though it is not likely after the question has been made. 

For our purposes, the nod/ shake detector should be as fast as possible. On the other hand, 
we assume that the nod/ shake input will be used only after the robot has asked something. 
Thus, the detector can produce nod/ shake detections at other times, as long as it outputs 
right decisions when they are needed. The major problem of observing the evolution of 
simple characteristics like intereye position or the rectangle that fits the skin-color blob is 
noise. Due to the unavoidable noise, a horizontal motion (the NO) does not produce a pure 
horizontal displacement of the observed characteristic, because it is not being tracked. Even 
if it was tracked, it could drift due to lighting changes or other reasons. In practice, a 
horizontal motion produces a certain vertical displacement in the observed characteristic. 
This, given the fact that decision thresholds are set very low, can lead the system to error. 
The performance can be even worse if there is egomotion, like in our case (camera placed on 
a head with pan- tilt). 

The proposed algorithm uses the pyramidal Lucas-Kanade tracking algorithm described in 
(Bouguet, 1999). In this case, there is tracking, and not of just one, but multiple 
characteristics, which increases the robustness of the system. The tracker looks first for a 
number of good points to track over the whole image, automatically. Those points are 
accentuated corners. From those points chosen by the tracker we attend only to those falling 
inside the rectangle that fits the skin-color blob, observing their evolution. Note that even 
with the LK tracker there is noise in many of the tracking points. Even in an apparently 
static scene there is a small motion in them. 

The method is shown working in Figure 9. The LK tracker allows to indirectly control the 
number of tracking points. The larger the number of tracking points, the more robust (and 
slow) the system. The method was tested giving a recognition rate of 100% (73 out of 73, 
questions with alternate YES/ NO responses, using the first response given by the system). 
What happens if there are small camera displacements? In order to see the effect of this, 
linear camera displacements were simulated in the tests. In each frame, an error is added to 
the position of all the tracking points. If (D x ,D y ) is the average displacement of the points 
inside the skin-color rectangle, then the new displacement is D x +e x and D y +e y . The error, 
which is random and different for each frame, is bounded by -e max < e x < e max and -e max < e y < 
e m ax- Note that in principle it is not possible to use a fixed threshold because the error is 
unknown. The error also affects to the tracking points that fall outside the rectangle. 
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Assuming that the objects that fall outside the rectangle are static we can eliminate the error 
and keep on using a fixed threshold, for (D x + e x ) - (F x + e x ) w D x and (D y +e y ) - (F y +e y ) « D y . 
For the system to work well it is needed that the face occupies a large part of the image. A 
zoom lens should be used. When a simulated error of e max =10 pixels was introduced, the 
recognition rate was 95.9% (70 out of 73). In this case there is a slight error due to the fact 
that the components F x and F y are not exactly zero even if the scene outside the rectangle is 
static. 




Fig. 9. Head nod/ shake detector. 

Another type of error that can appear when the camera is mounted on a mobile device like a 
pan-tilt unit is the horizontal axis inclination. In practice, this situation is common, 
especially with small inclinations. Inclinations can be a problem for deciding between a YES 
and a NO. In order to test this effect, an inclination error was simulated in the tests (with the 
correction of egomotion active). The error is a rotation of the displacement vectors D a 
certain angle a clockwise. Recognition rates were measured for different values of a, 
producing useful rates for small inclinations: 90% (60 out of 66) for a = 20, 83.8% (57 out of 
68) for a = 40 and 9.5% (6 out of 63) for a = 50. 

6. Habituation 

Habituation is a filtering mechanism that has received a lot of attention in physiology and 
psychology. In particular, some researchers have investigated the mechanisms of 
habituation in animals, being one of the most known works the study of the Aplysia's 
gillwithdrawal reflex (Castellucci et al., 1970). When the animal's siphon is touched, its gill 
contracts for a few seconds. If the siphon is stimulated repeatedly, the gill-withdrawal effect 
tends to disappear. Crook and Hayes (Crook & Hayes, 2001) comment on a study carried 
out on two monkeys by Xiang and Brown who identified neurons that exhibit a habituation 
mechanism since their activity decreases as the stimulus is shown repeatedly. Stanley's 
model (Stanley, 1976) of habituation, proposed to simulate habituation data obtained from 
the cat spinal cord, has been widely used in the literature. This model describes the decrease 
efficacy y of a synapsis by the first-order differential equation: 

T^ = a(y -y(t))-s(t) (7) 
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where yo is the normal, initial value of y, S(t) represents the external stimulation, x is a time 
constant that governs the rate of habituation and a regulates the rate of recovery. Equation 
(7) ensures that the synaptic efficacy decreases when the input signal S(t) increases and 
returns to its maximum yO in the absence of an input signal. 

The model given by (7) can only explain short-term habituation, so Wang introduced a 
model to incorporate both short-term and long-term habituation using an inverse S-shaped 
curve (Wang, 1995): 

r && = az(t) (y - y(t)) - Py(t)S(t) (8) 

^ = 7 z(t)(z(t)-05(t) (9) 

where a, yo and y have the same meaning than in (7), & regulates the habituation and z(t) 
decreases monotonically with each activation of the external stimulation S(t), and models 
the long-term habituation. Due to this effect of z(t) after a large number of activations, the 
recovery rate is slower. 

Note that novelty detection is a concept related to habituation. Novelty detection is the 
discovery of stimuli not perceived before and so habituation serves as a novelty filter (Stiles 
& Ghosh, 1995). From an engineering viewpoint, perceptual user interfaces, like humanlike 
robots, should be endowed with a habituation mechanism. The interest is twofold. First, it 
would be a filtering mechanism, discarding (or minimizing the importance of) repetitive 
information while paying attention to new experiences. This is in part motivated by the 
desire to distinguish between artificial and human signals. Artificial signals are often static 
or repeat with a fixed frequency. We do not want our robot to pay much attention to the 
hands of a wall-mounted clock. Instead, it would be more interesting to detect nonrepetitive 
stimuli, such as a conversation or a sudden loud noise. Note that we generally consider 
monotonous signals as those having a fixed frequency or frequencies (which can be zero, 
that is, the signal does not change) but signals whose frequency changes in a periodic 
pattern could also be considered monotonous. Higher scales are also possible but we do not 
consider them in this work because they are very hard to visualize and real examples of 
them are not so common. Second, habituation would lead to a more human-like behaviour, 
as perceived by users of the interface. As an example of this, consider Kismet. Someone can 
catch the eye of the system while waving a hand in its visual field of view, but if the 
stimulus is repetitive for a long time the system can show a lack of interest in it. Many 
aspects of Kismet's mental architecture are directly or indirectly influenced by the detection 
of monotonous sensory signals: stimulation and fatigue drives and the arousal dimension of 
its affect space (and in turn some emotional states, like surprise, boredom or interest). If we 
use the model of Equation (7) we can obtain undesired effects with certain stimuli. A 
periodic input signal (with frequency greater than zero) can produce a response that does 
not exhibit habituation. This is due to the fact that the model does not account for changing 
stimuli, but for continuous ones. In order to include this fact in the model, we propose to use 
an auxiliary signal which will be zero when the stimulus is stationary or with a fixed 
frequency, and one otherwise, and use this signal as an input to the habituation model (7). 
The auxiliary signal, which basically detects monotonous stimuli, is obtained from the 
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spectrogram of the stimulus itself. The spectrogram is a time-frequency distribution of a 
signal, and it is based on the Fourier Transform with a sliding window, see Figure 10 
(Holland et al, 2000). 





Fig. 10. Audio signal (left) and its corresponding spectrogram (right). 

Spectrograms are computed from windows of the input signal. These windows, of length 1, 
overlap by 1 - 1 samples. Let each spectrogram be represented as a matrix M, in which rows 
represent frequencies and columns represent time. We calculate the variance of each row of 
M, which produces a column vector v. The norm of this vector v is a measure of how 
monotonous the input signal is. The norm will be high when the signal is changing, and low 
otherwise. Thus, the auxiliary signal needed is simply the thresholded norm of v. The 
amplitude of the input signal affects the power content of the spectrograms, and in turn the 
norm of v. Thus, prior to calculating the FFT the input signal must be normalized dividing 
each input window by the sum of its absolute values. A value of 1 for the auxiliary signal 
will mean that there are changes in the input signal, while a value of indicates that the 
input signal is monotonous. Once the auxiliary signal is available, the model (7) is used to 
get the desired habituation behaviour, as controlled by parameters x and a. The auxiliary 
signal is then, for a given threshold T: 



■fi 



if \v\ > T 
if \v\ < T 



(10) 



With this method both static and fixed frequency stimuli can be detected. The algorithm 
described above was implemented to test it with different input signals. The first 
experiments that we present use only the first level mentioned above. 

In order to gather signals from the visual domain, we recorded video containing a yellow 
bright stimulus (a yellow card) that was moved in a repetitive fashion, see Figure 11-a). 
Using simple segmentation techniques we extracted the centroid of the card on each frame 
(384x288) and summed the x and y pixel coordinates to form the one-dimensional signal of 
Figure 5.29-b). The sequence of card movements throughout the recording was: horizontal 
movement, random (aperiodic) movement, vertical movement and vertical movement at a 
different frequency than the previous one. The results appear in Figure 12. Windows of 128 
samples were used, and the variance threshold was set to 1000. 
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The habituation mechanism described here was implemented in CASIMIRO, for signals in 
the visual domain only, i.e. images taken by the stereo camera (see Section 3). The difference 
between the current and previous frame is calculated. Then it is thresholded and filtered 
with Open and Close operators. Also, blobs smaller than a threshold are removed. Then the 
centre of mass of the resultant image is calculated. The signal that feeds the habituation 
algorithm is the sum of the x and y components of the centre of mass. This way when the 
image does not show significant changes or repetitive movements are present for a while the 
habituation signal grows. When it grows larger than a threshold, an inhibition signal is sent 
to the Attention module, which then changes its focus of attention. Neck movements 
produce changes in the images, though it was observed that they are not periodic, and so 
habituation does not grow. 





a) 



b) 



Fig. 11. a) Video recording used for the visual habituation experiment, b) unidimensional 
signal extracted from it. 





a) 



b) 



Fig. 12. a) Evolution of the (12) norm of the variance vector v, b) habituation level, using x : 
5, a = 1. 
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7. Other abilities 

In this section two minor abilities of the robot are described: an owner recognition method 
that is simple yet powerful enough and a "hand waving" detector. 
Owner recognition: 

The recognition of the robot's owner or caregiver is a potentially important feature. The 
recognition may not affect the creator at all, but it could impress other people present in the 
room. Person recognition is difficult to achieve, the identification of the robot's owner may 
be more feasible, for the problem reduces to two classes (owner and not-owner). Still, using 
face or voice for owner recognition would lead to essentially the same lack of robustness of 
face and speech recognition for multiple individuals. Who is the owner of the robot? The 
correct answer is: the person who buys it. However, we are more interested in the role of the 
person who cares the robot and uses it more frequently (this person is generally the owner 
too). This person is the one who switches the robot on, which is usually done from a certain 
part of the robot or from a certain computer. That cue may be exploited to recognize the 
robot owner or caregiver. Amazing Amanda, a doll released in autumn of 2005, is able to 
recognize the girl that takes the mother role. Once the doll is activated, it starts asking 
questions. That way, the doll can "learn" the girl's voice patterns. From that moment on, the 
doll is able to recognize the utterances of its "mommy". Other voices can lead Amanda to 
say "You don't sound like Mommy". 

Such technique may seem rather ad hoc. However, the approach finds striking examples in 
nature. Lorenz, one of the founders of ethology, found that, upon coming out of their eggs, 
geese follow and become attached to the first moving object that they encounter. He showed 
this by rearing the geese from hatching. From that moment on the geese would follow him. 
Such phenomenon, which also appears in mammals, is known as imprinting (Lorenz, 1981). 




Fig. 13. The computer from where CASIMIRO is started. The interaction space is on the 
right. 
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In the case of CASIMIRO, the main computer (from which the robot is switched on) is 
situated behind the robot, on the same table, see Figure 13. A camera was placed on top of 
that computer. The owner detection module uses that camera to search for a skin coloured 
blob in the image. When the robot is switched on this module will detect a skin coloured 
blob. The camera has a wide-angle lens, and a relatively low resolution of 160x120 is used. 
When no blob is encountered in the image the module notifies the Attention module of that 
event. At that moment the owner detection module exits in order to free CPU resources. 
Once it has been notified by the owner detection module, the Attention module considers 
the owner as the first blob that "enters" the omnidirectional camera image from the left. The 
"owner" property is stored along with the individual in the tracking process. This simple 
procedure is a form of imprinting. In a sense, the robot finds its owner-caregiver in the first 
human it sees. It does not stores any biometric features to recognize the owner after being 
switched on, only its position. 
Hand-waving detector: 

In initial tests with the robot we saw that people tended to wave his/her hand to the robot. 
We decided to detect this and make the robot respond with a funny phrase. The contour of 
each foreground blob is obtained and compared with the contour of the same blob in the 
previous frame. The comparison is made using Hu moments (Hu, 1962). When the 
comparison yields too much difference between the contours (i.e. a threshold is exceeded) a 
flag is activated. When the face detection module (Section 3) detects more than one skin 
colour blob in the image and a certain amount of motion and the flag is activated then the 
Boolean perception HANDWAVING is set to TRUE. 

8. Conclusion 

A number of simple but useful computer vision techniques have been described, suitable for 
human-robot interaction. First, an omnidirectional camera setting is described that can 
detect people in the surroundings of the robot, giving their angular positions and a rough 
estimate of the distance. The device can be easily built with inexpensive components. 
Second, we comment on a color-based face detection technique that can alleviate skin-color 
false positives. Third, a simple head nod and shake detector is described, suitable for 
detecting affirmative/ negative, approval/ disapproval, understanding/ disbelief head 
gestures. Fourth, a habituation-detection scheme has been described, allowing the robot to 
focus on natural signals in the environment. Finally, two minor owner recognition and 
hand-waving detection abilities are also described. All the techniques have been 
satisfactorily implemented and tested on a prototype social robot. 

Future work should focus on how computer vision could aid in hands-free speech 
recognition, an all-important ability for an interactive robot. Progress has been already made 
by syncing lip motion with audio speech signals, although the processing cost is still too 
long for useful real-time interaction. 
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1. Introduction 

In this chapter we will present the progress made in automatic speech recognition for 

Romanian language based on the ASRS_RL (Automatic Speech Recognition System for 

Romanian Language) research platform. 

Speech recognition is a research domain with a long history, but despite this fact, still open 

for new investigations and answers to the not yet finally solved questions. This situation can 

be explained by the difficulty of the task, underlying on the fact that speech is a human 

product, with a high degree of correlation in content, but with a great variability in the 

formal manifestation as an acoustic signal. Great difficulties cause also the imperfection of 

the audio chain and the noises in the environment. 

The best-known strategies for speech recognition are the statistical and the connectionist 

ones, but fuzzy sets can also play an important role. 

Based on HMM's the statistical strategies have many advantages, among them being 

recalled: rich mathematical framework, powerful learning and decoding methods, good 

sequences handling capabilities, flexible topology for statistical phonology and syntax. The 

disadvantages lie in the poor discrimination between the models and in the unrealistic 

assumptions that must be made to construct the HMM's theory, namely the independence 

of the successive feature frames (input vectors) and the first order Markov process. 

Based on artificial neural networks (ANNs), the connectionist strategies for speech 

recognition have the advantages of the massive parallelism, good adaptation, efficient 

algorithms for solving classification problems and intrinsic discriminative properties. 

However, the neural nets have difficulties in handling the temporal dependencies inherent 

in speech data. 

The learning capabilities of the statistical and the neural models are very important, 

classifier built on such bases having the possibility to recognize new, unknown patterns 

with the experience obtained by training. 

The introduction of fuzzy sets allows on one hand the so-called fuzzy decisions, on other 

hand the " fuzzy fication" of input data, often more suitable for recognition of pattern 

produced by human beings, by speaking, for example. In a fuzzy decision, the recognizer 

realizes the classification based on the degree of membership to a given class for the pattern 

to be classified, a pattern belonging in a certain measure to each of the possible classes. This 
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relaxation in decision leads to significant enhancements in recognition performances, 
situation that can also be obtained by looking in a fuzzy way to the input data 
The learning capabilities offered by the statistical and the connectionist paradigms and also 
a "nuanced" inside in the reality of the input and output domains of the speech recognizers 
contribute to a kind of "human likely " behaviour of this automata. 

These three main strategies were applied in our speech recognition experiments and the 
developed algorithms were incorporate in our research platform, the ASRS_RL. The 
statistical strategies played the most important role in the development of continuous 
speech recognition based on HMMs. The neural strategies are applied in form of multilayer 
perceptrons (MLP) and Kohonen maps (KM) for tasks like vowel or digit recognition. 
Algorithms for hybrid structures like fuzzy HMM, fuzzy MLP or MLP -HMM are also 
incorporated in our research platform. 

The chapter will be structured as: Section 2 will describe the "state of the art" of the 
techniques applied for Romanian language. The Section 3 is dedicated to the capabilities of 
the ASRS_RL system. The data bases for training and testing, the feature extraction methods 
and the learning strategies incorporated in the system are briefly introduced. In Section 4 
are shown and commented the experimental results of the ASRS_RL platform in two tasks: 
first a task of continuous speech recognition, next a telephone dial experiment. In the first task 
are applied HMMs with context dependent modelling and the performance is expressed in 
word recognition rate (WRR) for the recognition experiments and in phrase recognition rate 
(PRR) for speech understanding experiments. For the second task are implemented HMMs 
without context dependent modelling, for monophones. Section 5 presents results and 
comments for experiments done applying alternative strategies to HMM. These strategies are 
investigated for simple tasks like vowel and digit recognition. The error rate is evaluated for 
vowel recognized by using MLP, KM, SVM (Support Vector Machine), fuzzy - MLP and 
fuzzy-HMM strategies and for digits recognized by the HMM - MLP. 

The last Section (Section 6) end the chapter with conclusions extracted from the 
experimental results and with perspectives for our future work. 

2. Techniques applied for Romanian language 

A brief history of the speech processing techniques for Romanian language was given by the 
president of the Romanian Academy, Prof. Dr. ing. Mihai Draganescu in the introductory 
speech to the third conference dedicated to man-machine communication by natural 
languages, SPED 2003, held at Bucharest: "Almost 20 years ago, the Romanian Academy 
organized the first session on the analysis and synthesis of the speech signal. It was a moment of 
recognition of the activity of Romanian scientists in the domain of speech technology, with works 
beginning in 1963 (Edmon Nicolau, Inge Weber, Stefan Gavat), 1973 (Aurelian Lazaroiu), 1976 
(Eugeniu Oancea, with a volume on analysis and synthesis of speech) and with new papers of 
Corneliu Burileanu, Horia Nicolai Teodorescu, Eugeniu Oancea, Grigore Stolojanu, Virgil Enatescu 
and others. Since then the domain evolved from Speech technology to spoken language technology. 
This had to be foreseen however from the very beginning involving the use of artificial intelligence, 
both for natural language processing and for acoustic-phonetic processes of the spoken language/' 
(Draganescu, 2003). 

Continuing this tradition, our team has experimented a set of Computational Intelligence 
algorithms for Automatic Speech Recognition (ASR) extended also for speech 
understanding in the frame of the three main paradigms mentioned in the introduction. 
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We started caring out experiments with neural recognizers for vowels (Grigore et al., 1996), 
(Grigore et al., 1998), for isolated words (Valsan et al., 1998-a) and word spotting (Valsan et 
al., 1998-b). Then we introduced the HMM modelling in our experiments, first for isolated 
word recognition, further for continuous speech. The refinement of the HMM based 
recognizers was our permanent goal, and today we can report a system with reasonable 
performances in continuous speech recognition, for Romanian language ( Gavat et al. 2003, 
Dumitru et al„ 2006, Dumitru et al,2007, Gavat et al, 2008) 

But also very attractive in order to improve recognition performance was the development of 
hybrid systems. One of the firsts was a neuro-statistic hybrid, in different ways proposed by 
many authors (Bourlard et al., 1990), (Richard et al, 1991), (Lippmann et al., 1993), (Bourlard et 
al., 1994). We realized such a system for Romanian language as a variant consisting in a HMM 
assisted by a MLP as a posteriori probability estimator (Gavat et al., 1996-a). This combination, 
realized between components with complementary properties (i.e. the good integrating time 
successions HMM and the high discriminative MLP), leads to an increase in recognition rates 
that exceeds 2%. Furthermore, as suggested in (Juang et al., 1992), (Reichl et al., 1994) we 
refined our hybrid structure by a supplementary discriminative training (Gavat et al., 1998), 
obtaining a further improvement with around 3.5%. We applied also fuzzy variants for neural 
and hard classifiers. By applying fuzzy decisions, we have obtained improved performances in 
classical algorithms like k-NN or ISODATA (Gavat et al., 1996-a) and also in neural 
recognizers realized with MLP or self-organizing feature maps (Gavat et al., 1997), (Grigore et 
al., 1999). To apply fuzzy concepts to HMMs was the next natural step to be followed in our 
studies, experimenting in speech recognition the hybrid called Fuzzy-HMM (FHMM). (Gavat 
et al., 2001-a), (Gavat et al., 2001-b).This developed hybrid systems were applied until now 
only in tasks for vowel and isolated word recognition, the implementation for continuous 
speech recognition remaining as a future desiderate. 

3. ASRS_RL system 

Speech is a communication modality, so how the communication is accomplished is a 
question of high interest. Traditionally (Juanhg et al., 2000) it can be admitted that the 
speech communication chain is organized in four stages: detection of acoustic-phonetic cues 
to form words, syntactic and grammatical analysis to build sentences, semantic evaluation 
to determine the possible meanings of a sentence and pragmatic evaluation in order to select 
the convenient meaning. 

An ASR (Automatic Speech Recognition) system is a machine that performs in two stages 
(the first two) and an ASRU (automatic speech recognition and understanding) system is a 
machine for which all stage are necessary. But, a lot of work must still be done to acquire the 
whole knowledge necessary to entirely realize this last task. 

3.1 Capabilities of the ASRS_RL 

Automatic Speech Recognition System for Romanian Language (ASRS_RL) is a viable 
structure, offering multiple options for the tasks that can be performed, for the features that 
can be used to characterize speech frames, for the learning strategies applied, for the speech 
databases necessary in training and testing stage. It allows also multilingual (Romanian, 
English, German and can be extended to other languages) and multimodal (speech and 
image entries) experiments. One window of the MATLAB interface to choose the working 
options is shown in Fig.l 
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Fig. 1. The recognition system ASRS_RL 

a. The possible tasks that can be performed with this system are: continuous speech 
recognition; digit recognition and vowel recognition. 

b. The possible databases for training/ testing are: CDRL (Continuous Database for 
Romanian Language); DDRL (Digit Database for Romanian Language); DDEL (Digit 
Database for English Language); VDRL (Vowel Database for Romanian Language). The 
data are sampled by 16 kHz, quantified with 16 bits, and recorded in a laboratory 
environment (for CDRL, DDRL and VDRL) (Dumitru et al, 2007). For DDEL the 
database used is the AMP database (www.amp.ece. cmu.edu/_download 
/Intel/ feature_data.html). The DDRL and DDEL are available for three different SNR: 
19dB, 25dB and 30dB. 

c. The phonetic transcription for each database was created based on the SAMPA 
standard (Sampa). 

d. The features vectors to characterize speech frames (and the number of coefficients for 
each type) can be constituted (Huang at al., 2001) (Hermansky, 1990) by: formants, 
obtained from cepstral analysis; mel frequency cepstral coefficients (MFCC) with or 
without first (D-delta) and second (A-acceleration) order variations, obtained from 
perceptual cepstral analysis; linear prediction coefficients (LPC) obtained by linear 
prediction; perceptual linear prediction (PLP) coefficients obtained by perceptual linear 
prediction. To this features frame energy (E), zero crossing rate (Zo), and other 
information can be added. 

e. The possible learning strategies (Goronzy, S. (2002) that can be chosen in the system in 
order to build acoustical models are: hidden Markov models (HMM) for monophones 
(phonemes without context) and triphones (phonemes with left and right context) 
(Young, 1992); Support Vector Machines (SVM); artificial neural networks in form of 
multilayer perceptrons (MLP) and Kohonen maps (KM); hybrid structures like the 
neuro-statistical hybrid (HMM-MLP), or the fuzzy variants for the MLP or the HMM. 



Progress in Speech Recognition for Romanian Language 449 

3.2 Databases 

In this sub-section we present our databases for Romanian language with the principal 
characteristics, as well as a " state of the art" on the most important international databases. 
Based on the investigation of the existing standardized databases for different languages, 
the most important databases are: for English - TIMIT, WSJO, AMP, etc; for German - 
Vermobil; for French - NEOLOGOS, for Spanish - Entropic Latino40; for Hungarian - 
OASIS; but there are also some multilingual databases like CSLU, SpeechDat, Appen , etc. 
TIMIT Acoustic-Phonetic Continuous Speech Corpus was created by DARPA-ISTO 
(Defense Advanced Research Projects Agency - Information Science and Technology Office), 
and the text of this corpus was realized by MIT (Massachusetts Institute of Technology), SRI 
(Stanford Research Institute) and TI (Texas Instruments). 

The database contains 630 speakers from 8 region of the United States of America, 438 male 
speakers and 192 female speakers, each of them spoken 10 phrases (Timit). 
WSJO (Wall Street JOurnal) is the biggest and variat corpus for English, created by ARPA 
(Advanced Research Projects Agency). The phrases are from Wall street Journal. The initial 
number of words was 37 million, but finally reduced to 64000 words, the most frequent 
words in the journal. There are two corpora:J0 (84 speakers, 42 male speakers and 42 female 
speakers, 7193 phrases) and WSJ1 (200 speakers, 100 male speakers and 100 female speakers, 
29320 phrases) (Douglas et al, 1992); WSJCAM0 (for British English created at the 
Cambridge University in 1995; the corpus was selected from the Wall street Journal between 
1987-1989; the speakers are: between 18-23 years old, 46, between 24-28 years old, 30, 
between 29-40 years old, 7, more then 40 years old, 9; each of them uttered 110 phrases) 
(Robinson etal, 1995). 

AMP (Advanced Multimedia Processing) is a database created at the Carnegie Mellon 
University in Advanced Multimedia Processing Laboratory. The database is sampled by 
44,1kHz and quantified with 16 bits, mono system (Amp). 

The database contains 10 speakers (7 ale speakers and 4 female speakers) each spoken 78 
words for 10 times. 

CSLU (Centre of Spoken Language Understanding) is a database crated by Oregon Health 
& Science University. This database contains many corpora: 22 languages (a database for 22 
language); Alphadigits (3025 speakers uttered digits and stream of digits, 78044 audio files); 
Kids Speech (this database is created to help the children with hearing problems; about 100 
kids); Multilanguage Telephone Speech (for 11 language and contains 2052 speakers, spoken 
fixed phrases or free continuous speech); etc (Cslu). 

SpeechDat is created to develop systems for voice driven teleservice and speech interfaces. 
SpeechDat(M) is a database for 7 languages from West Europe (SpeDatM), 1000 speakers; 
SpeechDat(II) is for 11 languages, official languages of the Europeen Union, each of this 
databases contains between 500-5000 speakers; SpeechDat(E) is for 5 languages from Est 
Europe, contains between 1000-2500 speakers (SpeDatE). 

Appen is a collection of databases for 20 languages for different applications like: telephony, 
broadcast audio, desktop, pocket PC (Appen). The recorded files are realized in different 
location: car, studio, office, street or public place. We present the database for one 
application, telephony. The database is for spoken English in Australia, recorded in office 
environment. The first part of the database contains 500 speakers and immigrant speakers, 
each reading 165 phrases; the second part contains 1000 speakers (Australian speakers) 
reading 75 phrases. 
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Others databases are for spoken English in Canada (contains 49 speakers each uttered about 

99 phrases in office environment) and for spoken French in Canada (contains 48 speakers 

each spoken about 100 phrases) in office, car, home environment and public place. The 

phrases are constructed by digits, phonemes, names places jobs, answers yes / no), 

commands. 

Neologos was recorded on the French telephone network (Charlet et al., 2005) and two 

databases were obtained. The first database is IDIOLOGOS (with two sub-databases: 

Bootstrap - contains 1000 speakers (470 male speakers and 530 female speakers) with ages 

from 18 to 61 and up to 61, with a very good distribution, each of them uttered 45 phrases; 

Eingenspeakers - contains 200 speakers (97 male speakers and 103 female speakers) for 

different age, from 18 to 61 and up to 61, each of them spoken 45 phrases/ appeal telephonic, 

10 appeal are considered). 

The second database PAIDIALOGOS is for the children between 7 to 16 years, with one 

child having more than 16 years. The number of the children is 1010 (510 male children and 

500 female children), from different regions of France. The database contains: digits, 

numbers of credit cards, names of cities, etc. The data are sampled by 8 kHz, quantified with 

8 bits. 

Vermobil (Elra) was created between 1993-1996 by the research institutions and companies: 

Institut fur Phonetik und digitale Sprachverarbeitung (IPDS) - Kiel, Institut fur 

Kommunikation und Phonetik (IKP) - Bonn, Institut fur Phonetik und sprachliche 

Kommunikation - Miinchen and Universitat Karlsruhe. The database was divided in two 

databases: one for training (contains 12000 turns) and one for testing (1800 turns), with 

different dialects (12 dialects). One turn is equal with around 22.8 words. 

Entropic Latino40 (Entro40) was recorded in 1994, in Palo Alto - California, the speakers are 

Spanish, from Latino America. Database contains 5000 phrases, spoken by 40 speakers (20 

male speakers and 20 female speakers) each of them reading 125 phrases. The database is 

sampled by 16kHz, quantified with 16 bits and recorded in office environment. All the 

speakers are not specialized speakers and are between 18-59 years old. 

OASIS (Kocsor et al., 1995) was created at Hungarian Academy of Sciences (Research 

Group on Artificial Intelligence). The database contains speech files from 26 speakers (16 

male speakers and 10 female speakers) each of this speakers uttered the digits from 0-9, 

from 10-100, 1000, 1000000 and variants of composed numbers. The database is sampled by 

22,05kHz, quantified with 16 bits and recorded in office environment. 

Because in Romanian language no standard database is available, we create our database for 

our recognition tasks. The databases created are: OCDRL (Old Continuous Database for 

Romanian Language), CDRL (Continuous Database for Romanian Language), DDRL (Digit 

Database for Romanian Language) and VDRL (Vowel Database for Romanian Language). 

The data are sampled by 16 kHz, quantified with 16 bits, and recorded in a laboratory 

environment (Dumitru, 2006). 

OCDRL: for continuous speech recognition, is constituted by two databases: the training 

database contains 500 phrases, spoken by 10 speakers (8 males and 2 females), each speaker 

reading 50 phrases; the testing database contains 350 phrases spoken by the same speakers. 

The phrases from the database are for an application of telephone dial. 

A statistical investigation of the existing phonemes in the training database, based on 

phonetic transcription of the SAMPA standard (Sampa), is shown in the Table 1. We can see 

easy that the preponderant are the vowels (i, i_0, e, a, @, o, u, 1) and semi-vowels (j, e_X, w, 
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o_X). The distribution of phonemes is very similar to that given by linguists for Romanian 
language, so that we trusted the data base for the acoustical modelling. 
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Table 1. Number of phonemes existing in the training OCDRL database 

CDRL: for continuous speech recognition, the database is constituted for training by 3300 
phrases, spoken by 11 speakers (7 males and 4 females), each speaker reading 300 phrases, 
and for testing by 220 phrases spoken by the same speakers, each of them reading 20 
phrases. The phrases are from: information, telecommunication, geography, history, and 
sports domain. The training database contains over 3200 distinct words; the testing database 
contains 900 distinct words. 

In order to carry out our experiments the database was reorganized as follows: one database 
for male speakers (MS), one database for female speakers (FS) and one mixed database for 
male and female speakers (MS and FS). In all cases we have excluded one MS and one FS 
from the training and used for testing. 

Conducting the same statistical investigation of the phonemes distribution, (Table 2), we 
obtained very similar results with that obtained for the OCDRL data base. 
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Table 2. Number of phonemes existing in the training CDRL database 

DDRL: for digit recognition, the database contains speech data from 9 speakers (6 males and 
3 females) each speaker reading 9 digits (unu, doi, trei, patru, cinci, sase, sapte, opt, noua). 
VDRL: for vowel recognition, the database contains speech data from 19 speakers (9 males 
and 10 females) each reading the same 5 vowels (a, e, i, o, u). 

The database based on formants contains for the training 500 formant vectors, 100 for each 
vowel and for the testing 250 formant vectors, 50 for each vowel. 



3.3 Speech analysis 

First step in all recognition tasks is speech analysis, where the speech signal is processed in 
order to obtain important characteristics, further called features. By features extraction data 
compression is done, so that the amount of data used for comparisons is greatly reduced 
and thus, less computation and less time is needed for comparisons (Huang et al., 2001). 
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Our features extraction is based on perceptual cepstral coding and perceptual linear 
predictive coding, methods that will be presented further. 

The block scheme of the acoustical processor is shown in Fig. 2. Few blocks are common in 
both linear prediction and cepstral coding (Gavat et al., 2003). The first block in the scheme 
is the frame blocking, used because speech is fundamentally a non-stationary signal, so we 
cut short fragments of the speech signal, which are called frames and the speech is 
approximated as a quasi-stationary random process during a frame. Then we passed each 
frame through a Hamming window. We can compute at this time the energy of each frame 
and we can use the energy set of coefficients in the recognition process for more accuracy. 
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Fig. 2. The block scheme of the acoustical processor 

Cepstral analysis is a very reliable method to speech analysis and it can be realized 

applying FFT (Fast Fourier Transform) to the blocked and windowed time discrete signal 

s(n). After that, the modulus of the signal is calculated and the logarithm is taken, the result 

being proportional in fact to the power spectrum of the speech signal. Applying IFFT 

(Inverse Fast Fourier Transform) leads to the real cepstrum. 

Cepstral coefficients, are obtained by re-sampling of the real cepstrum; alone or in addition 

with the energy E, and/ or the first and second order differences constitutes a feature vector 

successfully applied in speech recognition 

Mel-frequency cepstral coefficients are calculated from the power spectrum after a spectral 

manipulation Sp.M.l in form of a filter bank that is a model for the critical band perception 

of the human cochlea. Often, in noisy environments the first (D-Delta) and second (A- 

Acceleration) order variations of the MFCC are applied to enhance the feature vector. 

Formants can be calculated from the smoothed spectrum, resulted applying FFT to the 

windowed cepstrum near the origin. 

Linear Predictive coding requires computation of the autocorrelation coefficients possible to 

be obtained according to the Wiener-Hinchin theorem, by applying IFFT to the power 

spectrum. 

Linear Prediction coefficients are obtained by low order all-pole modelling. The Levinson- 

Durbin recursive algorithm is used to solve the Yule-Walker equations. 
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where R(n) are the autocorrelation coefficients, and A(n) are the all-pole model coefficients 
(the predictor), and A (1) =2; through the conversion block the lasts can be converted in other 
types of coefficients like the reflection coefficients or the LPC cepstral coefficients. 
Perceptual Linear Prediction (PLP) coefficients are obtained by spectral manipulation 
Sp.M.2, displayed in Fig.3. The PLP analysis method (Hermansky, 1990) is more adapted to 
human hearing, in comparison to the classic Linear Prediction Coding because of the critical 
band processing of the analysed signal. The LP all-pole model approximates power 
distribution equally well at all frequencies of the analysis band. This assumption is 
inconsistent with human hearing, because beyond 800 Hz, the spectral resolution of hearing 
decreases with frequency; hearing is also more sensitive in the middle frequency range of 
the audible spectrum (Dumitru et al., 2006). 
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Fig. 3. Block representation for Sp. M. 2 
The power spectrum is computed as follows: 



P(co) = Re(SO)) 2 + ImCS^)) 2 
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The first step is a conversion from frequency to bark, which is a better representation of the 
human hearing resolution in frequency. The bark frequency corresponding to an audio 
frequency is: 
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The resulting warped spectrum is convoluted with the power spectrum of the critical band- 
masking curve, which acts like a bank of filters centred on Qj. 

The spectrum is pre-emphasized by an equal loudness curve, which is an approximation to 
the non-equal sensitivity of human hearing at different frequencies, at about 40dB level. A 
filter having the following transfer function gives the curve: 
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The last operation prior to the all-pole modelling is the cubic-root amplitude compression 
(Intensity - Loudness Conversion), which simulates the non-linear relation between the 
intensity of sound and its perceived loudness. Together with the psychophysical equal- 
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loudness pre-emphasis, this operation also reduces the spectral amplitude variation of the 
critical-band spectrum (Gavat et aL, 2008). 

3.4 Learning strategies 

In the same way into which a human learns to perceive speech from examples by listening, 
the automatic speech recognizer does apply a learning strategy in order to decode the 
pronounced word sequence from a sequence of elementary speech units like phonemes, 
with or without context. By learning are created models for each elementary speech unit, 
serving in the comparisons to make a decision about the uttered speech unit. We will 
describe further the learning strategies implemented in our research platform: hidden 
Markov models (HMM), artificial neural networks (ANN) in form of multilayer perceptrons 
(MLP) or Kohonen maps (KM). Some hybrid learning strategies are also implemented in 
form of a HMM- ANN combination and a fuzzy perceptron or fuzzy HMM. 
HMMs are finite automata, with a given number of states; passing from one state to another 
is made instantaneously at equally spaced time moments. At every pass from one state to 
another, the system generates observations, two processes taking place: the transparent one 
represented by the observations string (features sequence), and the hidden one, which 
cannot be observed, represented by the state string (Gavat et aL, 2000). 

In speech recognition, the left - right model (or the Bakis model) is considered the best 
choice. For each symbol, such a model is constructed; a word string is obtained by 
connecting corresponding HMMs together in sequence (Huang et aL, 2001). 
For limited vocabulary, word models are widely used, since they are accurate and trainable. 
In the situation of a specific and limited task they become valid if enough training data are 
available, but they are typically not generalizable. Usually for not very limited tasks are 
preferred phonetic models based on monophones (which are phonemes without context), 
because the phonemes are easy generalizable and of course also trainable. 
Monophones constitute the foundation of any training method and we also started with 
them (for any language). But in real speech the words are not simple strings of independent 
phonemes; these phonemes are affected for the immediately neighboring phonemes by co- 
articulation. This monophone models are changed now with triphone models (which are 
phonemes with context) that became actually the state of the art in automatic speech 
recognition for the large vocabularies (Young, 1992). 

A triphone model is a model that takes into consideration the left and the right context of 
the phonemes. Based on the SAMP A (Speech Assessment Methods Phonetic Alphabet) in 
Romanian language there are 34 phonemes; the number of necessary models (triphones) to 
be trained is about 40000, situation which is unacceptable. 

In the continuous speech recognition task we modelled only internal - word triphones and 
we adopted the state tying procedure, conducting to a controllable situation. If triphones are 
used in place of monophones, the number of needed models increases and it may occur the 
problem of insufficient training data. To solve this problem, tying of acoustically similar 
states of the models built for triphones corresponding to each context is an efficient solution 
(Fig.4). 

For example, in Fig. 4b four models are represented for different contexts of the phoneme 
"a", namely the triphones "k - a + S", "g - a + z" , "n - a+ )" , "m - a+ ]" . In Fig. 4c and 4d the 
clusters formed with acoustically similar states of the corresponding HMMs are represented. 
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Fig. 4. Different models for triphones around the phoneme "a". 

The choice of the states and the clustering in phonetic classes are achieved by mean of 
phonetic decision trees. A phonetic decision tree is a binary tree (Fig.5), where for each node 
of the tree questions are associated concerning the contexts of the phoneme (the number of 
the selected questions is 130 based on the knowledge about phonetic rules for our language). 
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Fig. 5. Phonetic tree for phoneme "m" in state 2. 
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The questions are chosen in order to increase the log likelihood of the data after splitting. 
Splitting is stopped when increasing in log likelihood is less than an imposed threshold. In 
the leaf nodes are concentrated all states having the same answer to the question made 
along the corresponding path. 

Fuzzy - HMM: The generalized model X = {A, B, 71) can be characterized by the same 
parameters (Huang et al., 2001) like the classical, well known model. The major difference in 
the fuzzy variant, is the interpretation of the probability densities for the classical HMM, as 
fuzzy densities. On this way the probabilistic similarity measure applied in the classical 
HMM is replaced by a more suitable fuzzy similarity measure. 

The succession of feature vectors, called the observation sequence, O, produces the state 
sequence S of the model, and, visiting for example at the moment t+1 the state q t+ i = S ,• , 

the symbol bj is generated. The corresponding symbol fuzzy density bj(O t ) measures the 

grade of certainty of the statement that we observed O t given that we are visiting state 

S J .To perform classification tasks, the fuzzy similarity measure must be calculated. Based 

on the fuzzy forward and backward variables, a fuzzy Viterbi algorithm is proposed in 
(Mahomed et al., 2000) for the case of the Choquet integral with respect to a fuzzy measure 
and multiplication as intersection operator. 

The fuzzy formulation of the forward variable a, bring an important relaxation in the 
assumption of statistical independence. 

The joint measure «q \jPi,...,O t }x \y A) can be written as a combination of two measures 
defined on C\, (>>,..., O^ and on the states respectively, no assumption about the 
decomposition of this measure being necessary, where Y = {yi,j>2>— >X/v} represent the 
states at time t+1 (Q is the space of observation vectors). 
For the standard HMM, the joint measure P(Oi,02,...,O t ,q t+ i = S A can be written as the 

product P(Oi,02,...,O t )-P(q t+ i = S A , so that two assumptions of statistical independence 

must be made: the observation at time t+1, O t+ \, is independent of the previous 

observations Oi,02,...,O t and the states at time t+1 are independent of the same 

observations, 0\ , O^ , • • • , O t . 

These conditions find a poor match in case of speech signals and therefore we hope in 

improvements due to the relaxation permitted by the fuzzy measure. 

Training of the generalized model can be performed with the re-estimation formulas also 

done in (Mahomed et al., 2000) for the Choquet integral. For each model we have trained 

with the re-estimation formulas the corresponding generalized models, GHMMs, with 3-5 

states, analog to the classical case. 

After the training, we have calculated the fuzzy measure P\0/ X), with the fuzzy Viterby 

algorithm and made the decisions for recognition in the same manner like for the classical 

HMM: the correct decision corresponds to the model for which the calculated measure has 

a maximum. 

MLP is the most common ANN architecture used for speech recognition. Typically, the 

classical MLPs have layered feed-forward architecture, with an input layer, one or more 
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intermediate (hidden) layers, and one output layer. The structure without hidden layer is 
called Boolean network and is a simple perceptron. 

Each layer computes a set of linear discriminative functions, followed by a non-linear 
function, which is often a sigmoid function. 

The numbers of neurons in the hidden layer was experimentally determined, trying to 
achieve an optimum between the following two opposite requirements: (a) lower computing 
volume and more rapid process of convergence in the learning period; (b) better 
performances from the correct classification of input patterns percentage. 
In the learning phase are determined the optimum values for weights (Goronzy, 2002), 
(Valsan et al., 2002) connecting the pairs of neurons from the adjoint layers in the input- 
output direction using the Back-Propagation algorithm. 

Introducing a fuzzy processing of the input layer of the MLP (fuzzy-MLP) is a solution to 
improve the MLP performances. 

The structure of the fuzzy neural network (Fig. 6) like the classical one is composed from an 
input layer, a hidden layer and an output layer. 



yi y2 y3 y 4 y 5 




output 
layer 

hidden 
layer 

fuzzy 
input 

input 
layer 



Fig. 6. Schematic diagram of fuzzy MLP 

First, the input values are described through a combination of 3 membership values in the 
linguistic properties set: low, medium and high. For doing this the ji membership function is 
used: 



7t(r,c,X) - 



2(l-\\r-c\\/iy forO<\\r-c\\<A/2 

\-2(\\r-c\\/A) 2 forO<\\r-c\\<A/2 

otherwise 



(5) 



where: A>0 is the radius of the n function with c as the central point, | | . | | denotes the 

Euclidian norm. 

For each component Fj t of the input vector Fj the parameters of the n membership 

function for each linguistic property: low (1), medium (m) and high (h) are computed using 
the relations: 
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/ ^m(Fij) = \^ji max — ^ ji mm.) ' ^ 
c m (Fij ) = r ji m [ n + A m (j7jj j 

h{Fji)={c m (Fji ) -Fji min ) / f dn 

(6) 
c \{Fji) = c m{Fji ) " Q^h(Fji) 

^h(Fji ) = \^ ji max — c m {Fji ))' J dn 

c h(Fji) = c m(Fji) + °' 5 h(Fji) 

where Fj imax , F jimm ^ denote the upper and lower bounds of the observed range of feature 

Fji and f^ is a parameter controlling the extent of overlapping. 

After this, the structure of the fuzzy neural network, like the classical one, is composed from 

a hidden layer and an output layer . 

The output vector is defined as the fuzzy class membership values. The membership value 

of the training Ff = (F^ F^ •••Fi n ) t to class C^ is computed using: 

Hk{Fi)-W + ^iklfd)) fc (7) 

where: fj , f c are constants controlling the amount of fuzziness in the class-membership 
set, Zj-£ is the weighted distance between the input vector Fj and the mean 

®k = (Okl ®k\ ■■■Ok\) t °f tne ^" tn c l ass / defined as: 



z ik = T[(Fji-O kj )/v k j] 2 (8) 

where: vy is the standard deviation of the;-th vectors' component from the C^ class. 

In the training stage, the Back-Propagation algorithm is used to determine the weights which 
minimized the mean square error (mse) between the real output d ,• and the desired one y ,• : 

mse= £ (Lidj-yj) 2 ) (9) 

7=1 
Fetrain 

Kohonen maps are competitive neural networks with topological character. The setting up 
of the winner neurons at output is done with keeping the topological relations between the 
input vectors. That is the reason for which this neural network is successfully used in 
pattern recognition (Fig. 7). 

It was applied the following structures to recognize vowels: (1) the input layer with 3 
neurons, corresponding to the three formant frequencies; (2) three case for the output layer: 
one-dimensional with 25 neurons, bidimensional with 4x4 neurons, 5x5 neurons, 6x6 
neurons, and toroidal with 25 neurons. 
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output layer 




input layer 



Xi x 2 x 3 . x m 

Fig. 7. Schematic diagram of Kohonen maps 

In the learning phase (Gold et aL, 2002) the structures are trained and the weights of the 
networks are established in two steps: (a) the determination of the winner neurons; (b) the 
adaptation of the weights for the winner neurons and for the neurons existing in a certain 
neighbourhood. In this step important are: the neighbourhood dimension r(t) decreasing 
during the training, and the learning rate rj(t) following in our experiments one of the laws: 

ri(t) = r l or r/(t) = t~ l/2 (10) 

Hybrid systems: the HMM-based speech recognition methods make use of a probability 
estimator, in order to approximate emission probabilities p(xr/qk), where x n represents the 
observed data feature, and q\ is the hypothesized HMM state (Rabiner, 1989). These 
probabilities are used by the basic HMM equations, and because the HMM is based on a 
strict formalism, when the HMM is modified, there is a great risk of losing the theoretical 
foundations or the efficiency of the training and recognition algorithms. Fortunately, a 
proper use of the MLPs can lead to obtain probabilities that are related with the HMM 
emission probabilities. 

In particular, MLPs can be trained to produce the a posteriori probability p(xr/qic), that is, the a 
posteriori probability of the HMM state given the acoustic data, when each MLP output is 
associated with a specific HMM state. Many authors have shown that the outputs of an 
ANN used as described above can be interpreted as estimates of a posteriori probabilities of 
output classes conditioned by the input, so we will not insist on this matter, but we will 
mention an important condition, useful for finding an acceptable connectionist probability 
estimator: the system must contains enough parameters to be trained to a good 
approximation of the mapping function between the input and the output classes. 
Thus, the a posteriori probabilities that are estimated by MLPs can be converted in emission 
probabilities by applying Bayes' rule (11) to the MLP outputs: 



P(x n /q k ) _p(qklx n ) 



(11) 



P(*n) P(<lk) 

That is, the emission probabilities are obtained by dividing the a posteriori estimations from 
the MLP outputs by estimations of the frequencies of each class, while the scaling factor 
p(x n ) is considered a constant for all classes, and will not modify the classification. 
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This was the idea that leads to hybrid neuro-statistical methods, that is, hybrid MLP - HMM 
methods (Fig. 8), applied for solving the speech recognition problem. 








Itt-ttMtt 



output 
layer 

hidden 
layer 

input 
layer 



Fig. 8. Schematic diagram of HMM-MLP 

4. Speech recognition using ASRS_RL system based on statistical strategies 

4.1 Continuous speech recognition 

To assess the performance of our ASRS_RL system we initiated comparative tests on the 
OCDRL and CDRL database for the word recognition rate (WRR) under different conditions 
for feature extraction, like perceptive cepstral analysis (MFCC_D_A-mel frequency cepstral 
coefficients with the corresponding first and second order variations -> having 36 
coefficients), and linear prediction (LPC-Linear Prediction Coding -> having 12 
coefficients). The evaluations are made in the case of acoustical model based on HMM with 
monophones (phonemes without context) (Dumitru, 2006). 

In order to carry out our experiments about speaker independence, the CDRL database was 
reorganized as follows: one database for male speakers (MS), one database for female 
speakers (FS) and one database for male and female speakers (MS&FS). In all cases we have 
excluded one MS and one FS from the training and used for testing. 

The words for both databases (OCDRL and CDRL) are chosen taking into account the 
characteristics of the Romanian language so that the phrase contains enough phonemes to 
create the models. 

The results obtained with the two databases are summarized in Table 3. After that, chosen 
the second database (CDRL) other evaluations are made. 





OCDRL 


CDRL 


MFCC_D_A 


LPC 


MFCC_D_A 


LPC 


Monophone 


96.10% 


73.80% 


57.44% 


26.10% 



Table 3. WRR for the two cases of databases, OCDRL and CDRL 

Trying to create a multi-domain recognition system, we extended the numbers of speakers 
and of spoken phrases for each speaker (extending OCDRL to CDRL) introducing 
vocabulary from other domains. In the same time we introduced for WRR evaluation new 
feature extraction methods, like PLP and new acoustical models based on the HMM with 
triphones (phonemes with context) (Young et al., 1994). 
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Table 4 shows the comparative results (average WRR) for three feature extraction methods, 
MFCC_D_A, LPC and PLP and for two modelling situations, based on monophones and 
triphones. 



Database 


Type 


MFCC_D_A 


LPC 


PLP 


CDRL 


Monophone 
Triphone 


57.44% 
78.24% 


26.10% 

51.50% 


47.00% 
70.11% 



Table 4. WRR for monophone and triphone modelling and for three feature extraction methods 

The next experiments performed on a Romanian language corpus prove that context- 
dependent models perform better than context-independent models. The recognition system 
was trained with 3000 phrases collected from ten speakers (more than 3000 distinct words). 
Gaussian output probability distribution was assumed for the 36 mel-frequency cepstrum 
coefficients (12 MFCC and first and second order variation). 

Firstly, the 34 context-independent models (monophones) were trained, and the system was 
tested with an unenrolled speaker. The testing utterances contained distinct words and a 
loop-grammar was assumed, i.e. any word could occur after any word, anytime. 
The core of our experiments is the construction of the decision tree for each state of the 
triphones derived from the same monophone. The monophones were cloned initially, and 
the resulted triphones were trained by embedded Baum-Welch procedure. Then, the 
decision tree was build for different thresholds (TL) in terms of log-likelihood resulting 
different size systems (example of the evolution of the log-probability values - in the 
training phase is presented in Fig. 9). The results are presented in Table 5. For a small 
threshold (TL) of 300, the trees are big and the system is large having 2954 tied states with a 
huge number of parameters. For a big threshold of 6000, the trees are much smaller, 
implying a great reduction in the system size, from 7521 triphone states to 416 states, (5.5% 
remained size) while the performance is degrading with less than 1 % . In Table 5, are given 
also the word recognition rate (WRR), the accuracy. 



TL 


Initial states / final states 


Remained size 


WRR 


300 


7521 / 2954 


39.30% 


90.14% 


900 


7521 / 1448 


19.30% 


89.60% 


1200 


7521 / 1164 


15.50% 


90.31% 


1800 


7521 / 908 


12.10% 


90.51% 


2400 


7521 / 747 


9.90% 


90.02% 


3000 


7521 / 643 


8.50% 


89.97% 


3600 


7521 / 573 


7.60% 


90.07% 


4200 


7521 / 522 


6.90% 


89.79% 


4800 


7521 / 480 


6.40% 


89.60% 


5400 


7521 / 446 


5.90% 


88.85% 


6000 


7521 / 416 


5.50% 


88.75% 



Table 5. The results obtained for different thresholds for constructing the phonetic trees 

Now, we evaluate the WRR for the CDRL database taking into account the following 
situations: triphone modelling/ monophone modelling; gender based training/ mixed 
training and LPC and PLP/ MFCC coefficients. 

The speech files from these databases were analysed in order to extract the interesting 
features. The feature extraction methods used are based on LPC, PLP and MFCC. 
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The results obtained in the experiments realized under these conditions are summarized in 
the next three Tables. 
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Fig. 9. Log-probability evolutions 



Training MS 


Type 


WRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


56.33% 


30.85% 


34.02% 


Triphone 


81.02% 


49.73% 


68.10% 


Testing FS 


Monophone 


40.98% 


23.23% 


25.12% 


Triphone 


72.86% 


47.68% 


59.00% 



Table 6. WRR training MS and testing MS and FS 



Training FS 


Type 


WRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


53.56% 


26.72% 


23.78% 


Triphone 


69.23% 


49.73% 


53.02% 


Testing FS 


Monophone 


56.67% 


31.11% 


34.22% 


Triphone 


78.43% 


61.15% 


58.55% 



Table 7. WRR training FS and testing MS and FS 



Training MS 
and FS 


Type 


WRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


57.44% 


26.10% 


47.00% 


Triphone 


78.24% 


51.50% 


70.11% 


Testing FS 


Monophone 


49.89% 


24.06% 


41.22% 


Triphone 


74.95% 


50.49% 


69.65% 



Table 8. WRR training MS&FS and testing MS and FS 
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The following comments can be made: 

a. for 12 LPC coefficients the word recognition rates are low: 30.85% (monophone) 
training and testing with MS and 49.73% (triphone); 31.11% (monophone) training and 
testing with FS and 61.15% (triphone); 26.10% (monophone) training MS and FS and 
testing with MS and 51.5% (triphone); 

b. for 5 PLP coefficients the obtained results are very promising, giving word recognition 
rates about 58.55% (triphone training and testing FS), 68.10% (triphone training and 
testing MS) and 70.11% (triphone training MS and FS and testing MS); 

c. for 36 MFCC_D_A coefficients (mel-cepstral coefficients with first and second order 
variation) we obtained the best results, as we expected: monophone 56.33% and 
triphone 81.02%, training and testing with MS; monophone 56.67% and triphone 
78.43%, training and testing with FS; monophone 57.44% and triphone 78.24%, training 
MS and FS and testing with MS. 

In the following part (ASRU system) (Juanhg et al., 200) are displayed the results obtained 
for PRR (Phrase Recognition Rate) in the same condition as well as evaluate the WRR: the 
CDRL database; the MFCC with the first and second order variation, or the PLP or the LPC 
feature vector; the acoustical model without and with context. 

The experimental results for PRR are shown in the next Tables, using for the word 
recognition order a loop grammar. 



Training MS 


Type 


PRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


20.00% 


5.00% 


10.00% 


Triphone 


66.25% 


11.25% 


37.50% 


Testing FS 


Monophone 


21.25% 


6.25% 


16.25% 


Triphone 


48.75% 


12.50% 


32.50% 



Table 9. PRR training MS and testing MS and FS 



Training FS 


Type 


PRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


21.25% 


3.75% 


6.26% 


Triphone 


37.50% 


11.25% 


21.25% 


Testing FS 


Monophone 


30.00% 


12.50% 


22.50% 


Triphone 


57.50% 


33.75% 


30.00% 



Table 10. PRR training FS and testing MS and FS 



Training MS 
and FS 


Type 


PRR 


MFCC_D_A 


LPC 


PLP 


Testing MS 


Monophone 


23.75% 


3.75% 


18.75% 


Triphone 


60.00% 


10.00% 


36.25% 


Testing FS 


Monophone 


31.25% 


6.25% 


20.00% 


Triphone 


55.00% 


11.25% 


46.25% 



Table 11. PRR training MS&FS and testing MS and FS 
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After the experiments made on continuous speech recognition (Dumitru, 2006) we have 
following observations: 

a. the triphone modelling is effective, conducting to increasing in WRR between 15% and 
30% versus the monophone modelling. The maximal enhancement exceeds 30% for 
training MS and testing FS for MFCC_D_A (Fig. 10a). 

b. A gender based training is conducting to good result for tests made with speakers from 
the same gender (training MS / testing MS: 81.02%, testing FS: 72.86%; training 
FS/ testing FS: 78.43%, testing MS: 69.23%); changing gender in testing versus training 
leads to a decrease in WRR around 10%. For a mixed trained data base changing 
gender determines only variations around 5% in WRR (Fig. 10b). 

c. The PRR variation follows the WRR line, but the results are drastically lower when we 
compare with WRR. 



WRR differencies (%) triphone/monophone 




□ training MS/testing 
MS 

■ training MS/testing 
FS 

□ training FS/testing 
MS 

□ training FS/testing 
FS 

■ training 
mixed/testing MS 

□ training 
mixed/testing FS 



a) 



WRR (%) triphone 




□ training MS/testing 
MS 

■ training MS/testing 
FS 

□ training FS/testing 
MS 

□ training FS/testing 
FS 

■ training 
mixed/testing MS 

□ training 
mixed/testing FS 



b) 



Fig. 10. WRR triphone vs. monophone 
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4.2 Telephone dial application 

Other investigations using HMM are done for telephone dial domain. For this application 

the HMM without context modelling and MFCC coefficients are considered. 

Our system was developed for a telephone numbers dialling application (Fig.ll). The 

training data are constituted by the 850 phrases (OCDRL database), spoken by ten speakers, 

and the testing data are constituted by the 30 phrases, spoken by in training enrolled and 

also unenrolled speakers. The data are in the typical "wave" format, sampled by 16 kHz, 

quantified with 16 bits, recorded in a laboratory environment. 

Examples of training phrases are: 

Formeaza opt sapte sase cinci sase patru doi trei; 

Telefoneaza Octavian; 

Formeaza opt sapte cinci patru trei doi unu. 

Examples of testing phrases are: 

Formeaza sapte doi opt cinci unu sase patru cinci unu unu; 

Telefoneaza Octavian Dumitru. 

In the first experiments the speech signal was parameterized with 39 coefficients 

(MFCC_D_A): 13 mel cepstral coefficients; 13 delta coefficients; 13 accelerations coefficients. 

The recognition rates are summarized in Table 12. 



Speaker 


No. test words 


No. recognised words 


Recognition rate 


Enrolled 


103 


98 


95.1% 


Unenrolled 


103 


93 


90.3% 



Table 12. Recognition rate 



Ij Recunoastere vocala 



Setari 








Fig. 11. Telephone dial application 

For the enrolled persons the recognition rate was 95.1%, and for the unenrolled speakers the 
recognition rate was lower as in the first case, but still quite satisfactory (Dumitru, 2006). 
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Our next experiments were conducted for different choices in speech parameterization. The 
experiments made in the further described conditions lead to the recognition rates showed 
for the corresponding parameters in Fig. 12. The chosen parameter sets are the following: 
MFCC_0 (13) - 13 mel cepstral coefficients; MFCC_0_E (14) - 13 mel cepstral coefficients with 
log energy; MFCC_0_D (26) -13 mel cepstral coefficients and 13 delta coefficients; 
MFCC_0_D_A (39) -13 mel cepstral coefficients, 13 delta coefficients and 13 acceleration 
coefficients; MFCC_D_A_E (39) -12 mel cepstral coefficients, 12 delta coefficients, 12 
acceleration coefficients, log energy, delta energy, acceleration energy. 

The best rate, 96%, was obtained with 13mel-cepstral coefficients and 13 delta coefficients 
(MFCC_0_D). 

Now, our interest is in medical domain to help the dentists in their job creating a system for 
completing the examination and treatment dental charts (a project between our laboratory 
and the University of Medicine and Pharmacy of Bucharest). 



Recognition rate 



98% 
96% 
94% 
92% 
90% 
88% 
86% 
84% 
82% 
80% 



^ 



■ 1 3 coeff 

■ 14 coeff 

□ 26 coeff 

□ 39 coeff 

■ 39 coeff 



r<f 



Fig. 12. Recognition rate for different number of coefficients 



5. Digit and vowel recognition using ASRS_RL system based on neural and 
hybrid strategies 

5.1 Digit recognition using neuro-statistical strategies 

In the first digit recognition experiment, we compare the performance of two kinds of 
classifiers, namely the SVM (only to compare with other structure) and HMM, the 
performance being appreciated by their recognition rate and by their generalization 
capacity. For that, we performed two types of tests on the DDRL database: first, with 
enrolled speakers, which mean that the speakers were involved both in training and testing. 
We used five speakers for training and four for testing. For the second type of tests, with 
unenrolled speakers we used the leave-one-out method: for each word, we trained the 
classifier with 8 speakers and tested with the 9th repeating the procedure for each speaker. 
The conditions for feature extraction are: perceptive cepstral analysis giving a 13 - 
dimensional vector having as components 13 MFCCs and perceptual linear prediction 
giving a 5 - dimensional feature vector having as components 5 PLP coefficients. 
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In the second digit recognition experiment, we evaluate the performance obtained with 
HMM and with the hybrid neuro-statistical system (HMM - MLP) for unenrolled and 
enrolled speaker (Dumitru, 2006). 

The digit parameters were extracted by cepstral analysis, in form of 12 mel-frequency 
cepstral coefficients. 

a. HMM: for each digit we constructed a left-right hidden Markov model with 3 states; 

b. HMM - MLP: the system consists of 9 hybrid models corresponding to 9 digits. Each 
hybrid model is made of 5 states, each state being associated with one output node of 
the MLP. The MLP has one hidden layer (100 nodes), and the input layer consisting of 
12 nodes (Dumitru et al, 2007). 

The results for digit recognition are presented in Table 13. 

The performance of the HMM, SVM and hybrid system are validate using other databases, 
for example Advanced Multimedia Laboratory from the Carnegie Mellon University. The 
results show the same behaviour, but the WRRs are slightly higher for the English database, 
recorded in a studio with specialized speakers. 



Type 


Features extraction 


Enrolled 


Unenrolled 


SVM 


MFCC 


97.70% 


91.70% 


HMM 


MFCC 


98.00% 


97.50% 


HMM-MLP 


MFCC 


98.50% 


98.30% 


SVM 


PLP 


91.70% 


84.70% 


HMM 


PLP 


95.10% 


94.20% 



Table 13. WRR for digit recognition using SVM, HMM and HMM-MLP 

Based on the experimental results obtained, the following conclusions can be extracted: 

a. It is to seen that SVM performance is slightly after that of the HMM, but is really 
promising, taking into account that the HMM has the benefit of a so long refinement 
time. 

b. Trying to reduce these limitation effects of HMMs (the model training is not 
discriminative), chosen an alternative approaches can be a good solution. This 
approaches combine the HMM with MLP into a hybrid system. The results for HMM 
are lower than the results for the hybrid system. 



5.2 Vowel recognition using neural and fuzzy strategies 

The learning strategies applied in our recognition experiments are: the Kohonen maps, the 
MLP, the fuzzy-MLP and the fuzzy-HMM. 
There are three experiments can be made: 

(A) In the first experiment, the vowel recognition rate using the VDRL database and the 
MFCC coefficients (in form of 12 mel-frequency cepstral coefficients) was determined; the 
results obtained with MLP and HMM as learning strategies are comparatively presented in 
Table 14. 

The VDRL database was organized as follows: one database for male speakers (MS), one 
database for female speakers (FS). In booth cases one male speaker (MS) and one female 
speaker (FS) was excluded from the training database and used their data for the testing. 
The experimented MLP is a two-layer perceptron trained with Back-Propagation algorithm, 
having in the output layer 5 nodes corresponding to the 5 vowels to be classified and 100 
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nodes in the hidden layer (experimentally chosen). The number of the input nodes is equal 

to the number of features (12 MFCC). 

The HMMs chosen for comparison are Bakis (or left-right) structures with five states and for 

each vowel one model is created (Gavat et al., 2000). 

In Table 14, are displayed only the results in the case of training MS and testing with MS 

and FS. Similarly results were obtained for the training with FS (Dumitru et al., 2007). 



Vowel 


HMM 


MLP 


MS 


FS 


MS 


FS 


a 


100.00% 


80.71% 


100.00% 


82.28% 


e 


85.81% 


43.25% 


94.82% 


50.67% 


i 


85.71% 


85.71% 


95.15% 


92.41% 


o 


90.90% 


51.33% 


97.00% 


52.78% 


u 


88.88% 


71.42% 


94.83% 


77.85% 


Mean 


91.26% 


66.48% 


96.36% 


72.20% 



Table 14. Vowel recognition rate in the case of training with MS and testing with MS and FS. 

(B) In the second experiment, the vowels were described by three formant frequencies and 

the error rates obtained with different learning strategies are given in Table 15. 

The database for formants contains 500 formant vectors, 100 for each vowel for the training 

and 250 formant vectors, 50 for each vowel for the testing. 

The learning structures applied for these investigations are: (a) KM with the input layer with 

3 neurons, corresponding to the three formant frequencies and three variants for the output 

layer: unidimensional with 25 neurons, bidimensional with 5x5 neurons, and toroidal with 

25 neurons; (b) MLP with 3 layers organized as it follows: (1) the input layer with 3 neurons, 

corresponding to the three formant frequencies; (2) the hidden layer with (Boolean 

network) or 4 neurons, (3) the output layer with 5 neurons corresponding each to a 

processed class (in our case the vowels a, e, i, o, u); (c) fuzzy-MLP. 



Vowel 


KM 1-dim 


KM 2-dim 


KM toroidal 


Boolean 


MLP 


Fuzzy MLP 


a 


2.60% 


1.20% 


2.00% 


4.00% 


0.00% 


1.50% 


e 


3.20% 


2.40% 


2.40% 


6.00% 


2.50% 


1.00% 


i 


2.20% 


1.60% 


1.20% 


4.00% 


1.00% 


0.50% 


o 


1.80% 


1.20% 


1.20% 


10.00% 


1.00% 


1.00% 


u 


2.20% 


2.10% 


1.80% 


10.00% 


1.00% 


0.00% 


Mean 


2.40% 


1.70% 


1.72% 


6.80% 


1.10% 


0.80% 



Table 15. Error rates in for different learning strategies for the case of formant 

(C) In the third experiment the parameterization is realized with the mel cepstral 
coefficients and the first and second order differences of these coefficients deduced from 
homomorfic filtering. The error rates obtained in the vowel recognition tests are given in 
Table 16, comparatively for the generalized HMM (fuzzy-HMM) and the classical HMM. 
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Vowel 


Fuzzy - HMM 


Classical HMM 


a 


5.10% 


6.90% 


e 


2.40% 


4.80% 


i 


3.80% 


7.30% 


o 


2.50% 


5.90% 


u 


0.70% 


3.90% 


Mean 


2.90% 


5.78% 



Table 16. Error rates for generalized and for classical HMMs 

For vowel recognition in Romanian language the following conclusion can be reported: 

(A) The recognition rates in the case of MLP are higher than in the case of HMM. A possible 
explanation can be the fact that the model training is discriminative, while in the case of 
HMM the training is not discriminative, which represents a disadvantage of HMM 
utilization. 

(B) The KM 2-dimensional structures and the toroidal have the same performance, weaker is 
the performance of the 1-dimensional structure. The best balanced situation corresponds to 
the 2-dimensional map 5x5, in which all neurons are associated to a vowel to be recognized. 
The performance obtained in the case of the Boolean network is unacceptable, but the MLP 
acts well. 

Using fuzzy-MLP structure it is an improvement with a mean value of 0.30% comparative 
with the non-fuzzy structure. 

(C) A mean decreasing of nearly 3% is realized in the error rate by adopting the fuzzy-HMM 
instead of the probabilistic one. 



6. Conclusions 

In this chapter we presented the work done until now to implement a tool for research in 
speech recognition for Romanian language. What we realized can be summarized as 
follows: 

• We have implemented basic algorithms for feature extraction for perceptual and simple 
cepstral analysis and perceptual and simple linear prediction 

• We have implemented learning strategies based on HMMs, ANNs, and hybrid 
strategies like fuzzy-MLP, fuzzy HMM or HMM-MLP in the framework of the 
statistical, connectionist and fuzzy paradigms of Computational Intelligence 

• We have constructed databases to train the models and to test our recognizers that obey 
the same statistical rules as given from linguists for Romanian language. 

• We have trained the models and tested the recognizers on this data bases 
What we achieved and what is further to be done is sketched bellow: 

• The obtained results in ASR for WRR are acceptable, what can not be said for ASRU 
experiments, so that much work must be done to enhance PRR by extending the 
knowledge resources to the semantic and pragmatic levels. Our resources for the 
moment cover the acoustic and phonetic level by the trained models and the dictionary 
and in a very primitive manner the syntactic level, by a loop grammar. A more complex 
grammar and a language model are our next objectives 
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• For validation of our algorithms, a professional data base for Romanian language could 
be very helpful. Some universities in Romania try to do that by cooperation and we 
hope to succeed in our attempt. 

• To extend the possibilities of our platform we are planning to enhance the feature 
extraction methods and also try to develop neural and hybrid strategies suitable for 
continuous speech recognition experiments 
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